A  COMPARISON  OF  METHODS  FOR  COMBINING 
TESTS  OF  SIGNIFICANCE 


BY 

WILLIAM  C.  LOUV 


A  DISSERTATION  PRESENTED  TO  THE  GRADUATE  COUNCIL 
OF  THE  UNIVERSITY  OF  FLORIDA 
IN  PARTIAL  FULFILLMENT  OF  THE  REQUIRE^fENTS  FOR  THE 
DEGREE  OF  DOCTOR  OF  PHILOSOPHY 


UNIVERSITY  OF  FLORIDA 
1979 


TO   JILL 


ACKNOWLEDGMENTS 

I  am  indebted  to  Dr.  Ramon  C.  Littell  for  his  guidance  and 
encouragement,  without  which  this  dissertation  would  not  have  been 
completed.   I  also  wish  to  thank  Dr.  John  G.  Saw  for  his  careful 
proofreading  and  many  helpful  suggestions.   The  assistance  of  Dr. 
Dennis  D.  Wackerly  throughout  my  course  of  graduate  study  is  greatly 
appreciated. 

My  special  thanks  go  to  Dr.  William  Mendenhall  who  gave  me  the 
opportunity  to  come  to  the  University  of  Florida  and  who  encouraged 
me  to  pursue  the  degree  of  Doctor  of  Philosophy. 


Ill 


TABLE  OF  CONTENTS 

Page 

ACKNOWLEDGMENTS  iii 

ABSTRACT vi 

CHAPTER 

I    INTRODUCTION  AND  LITERATURE  REVIEW   1 

1.1  Statement  of  the  Combination  Problem  1 

1.2  Non-Pararaetric  Combination  Methods  2 

1.3  A  Comparison  of  Non-Parametric  Methods  5 

1.4  Parametric  Combination  Methods  8 

1.5  Weighted  Methods  of  Combination   11 

1.6  The  Combination  of  Dependent  Tests 12 

1.7  The  Combination  of  Tests  Based  on  Discrete  Data   ...  13 

1.8  A  Preview  of  Chapters  II,  III,  and  IV 18 

II    BAHADUR  EFFICIENCIES  OF  GENERAL  COMBINATION  METHODS  ....  19 

2.1  The  Notion  of  Bahadur  Efficiency 19 

2.2  The  Exact  Slopes  for  T^'^^  and  T^^"* 21 

2.3  Further  Results  on  Bahadur  Efficiencies   26 

(F) 

2. A   Optimality  of  T    in  tlie  Discrete  Data  Case 28 

III    THE  COMBINATION  OF  BINOMIAL  EXPERIMENTS  32 

3.1  Introduction 32 

3.2  Parametric  Combination  Methods  33 

3.3  Exact  Slopes  of  Parametric  Methods  37 

3.4  Approximate  Slopes  of  Parametric  Methods  44 

3.5  Powers  of  Combination  Methods   54 

3.6  A  Synthesis  of  Comparisons 57 

(F) 

3.7  Approximnt ion  of  the  Null  Distributions  of  T    , 

T^'''^\  T^'^'-'^) 79 

IV    APPLfCATIONS  AND  FUTURE  RESEARCH   96 

4.1  Introduction 96 

4.2  Estimation:   Confidence  Regions  Based  on 
Non-parametric  Combination  Methods  96 

iv 


TABLE  OF  CONTENTS  (Continued) 

CHAPTER  IV  (Continued)  Page 

4.3  The  Combination  of  2  x  2  Tables 110 

4.4  Testing  for  the  Heterogeneity  of  Variances 113 

4.5  Testing  for  the  Difference  of  Means  with 

Incomplete  Data 115 

4.6  Asymptotic  Efficiencies  for  k^°° 115 

BIBLIOGRAPHY ng 

BIOGRAPHICAL  SKETCH  122 


Abstract  of  Dissertation  Presented  to  the  Graduate  Council 
of  the  University  of  Florida  in  Partial  Fulfillment  of  the  Requirements 
for  the  Degree  of  Doctor  of  Philosophy 


A  COMPARISON  OF  METHODS  FOR  COMBINING 
TESTS  OF  SIGNIFICANCE 

By 

William  C.  Louv 

August  1979 

Chairman:   Ramon  C.  Littell 
Major  Department:   Statistics 

Given  test  statistics  X   ,...,X    for  testing  the  null 
hypotheses  H  ,...,H  ,  respectively,  the  combining  problem  is  to  select 
a  function  of  X   ,...,X    to  be  used  as  an  overall  test  of  the  hypoth- 
esis H  =  H  "^  H  '^    •••  1^  ^Ik.  •   Functions  based  on  the  probability  integral 
transformation,  that  is,  the  significance  levels  attained  by  X   ,...,X   , 
form  a  class  of  non-parametric  combining  methods.   These  methods  are  com- 
pared in  a  general  setting  with  respect  to  Bahadur  asymptotic  relative 
efficiency.   It  is  concluded  that  Fisher's  omnibus  method  is  at  least  as 
efficient  as  all  other  methods  whether  X   ,...,X    arise  from  contin- 
uous or  discrete  distributions. 

Given  a  specific  parametric  setting,  it  may  be  possible  to 
improve  upon  the  non-parametric  methods.   The  problem  of  combining  binom- 
ial experiments  is  studied  in  detail.   Parametric  methods  analogous  to 
the  sum  of  chi's  procedure  and  the  Cochran-Mantel-Haenszel  procedure  as 
well  as  the  likelihood  ratio  test  and  an  approximate  likelihood  ratio 
test  are  compared  to  Fisher's  method.   Comparisons  are  made  with  respect 
to  Bahadur  efficiency  and  with  respect  to  exact  power.   The  power 

vi 


comparisons  Cake  tlie  form  of  plots  of  contours  of  equal  power.   If 
prior  information  concerning  the  nature  of  the  unknown  binomial  success 
probabilities  is  unavailable,  Fisher's  method  is  recommended.   Other 
methods  are  preferred  when  specific  assumptions  can  be  made  concerning 
the  success  probabilities.   For  instance,  the  Cochran-Mantel-Haenszel 
procedure  is  optimal  when  the  success  probabilities  have  a  common  value. 
Fisher's  statistic  has  a  chi-square  distribution  with  2k  degrees 
of  freedom  when  X    ,...,X    are  continuous.   In  the  discrete  case, 
however,  the  exact  distribution  of  Fisher's  statistic  is  difficult  to 
obtain.   Several  approximate  methods  are  compared  and  Lancaster's  mean 
chi-square  approximation  is  recommended. 

The  combining  problem  is  also  approached  from  the  standpoint 
of  estimation.   Non-parametric  methods  are  inverted  to  form  k-diraensional 
confidence  regions.   Several  examples  for  k  =  2  are  graphically  displayed.. 
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CHAPTER  I 


INTRODUCTION  AND  LITERATURE  REVIEW 


1.1   Statement  of  the  Combination  Problem 


The  problem  of  combining  tests  of  significance  has  been  studied 

by  several  writers  over  the  past  fifty  years.   The  problem  is:   Given 

test  statistics  X    ,...,X    for  testing  null  hypotheses  H  ,...,H  , 

respectively,  to  select  a  function  of  X   ,...,X    to  be  used  as  the 

combined  test  of  the  hypothesis  H  =  H,  n  H.  n  . . .  n  H,  .   In  most  of  the 

12         k 

work  cited,  the  X    are  assumed  to  be  mutually  independent,  and,  except 
where  stated  otherwise,  that  is  true  in  this  paper. 

Some  practical  situations  in  which  an  experimenter  may  wish  to 
combine  tests  are: 

i.   The  data  from  k  separate  experiments,  each  conducted  to 
test  the  same  H,  yield  the  respective  test  statistics  X    ,...,X 
It  is  desired  to  pool  the  information  from  the  separate  experiments  to 
form  a  combined  test  of  H.   It  would  be  desirable  to  pool  the  informa- 
tion by  combining  the  X    if  (a)  only  the  X  ^  ,  instead  of  the  raw 
data,  are  available,  if  (b)  the  information  from  the  i    experiment  is 
sufficiently  contained  in  X    ,  or  if  (c)  a  theoretically  optimal  test 
based  on  all  the  data  is  intractible. 

ii.   The  i   of  k  experiments  yields  X    to  test  a  hypothesis 
H.,  i  =  1 k,  and  a  researcher  wishes  to  simultaneously  test 


the  truth  of  H  ,...,11  .   Considerations  (a),  (b)  ,  and  (c)  in  the  preced- 
ing paragraph  again  lead  to  the  desirability  of  combining  X    as  a  test 

of  H  =  H,  n  ...  n  H,  . 
1  k 

iii.  A  simultaneous  test  of  H  =  H  n  ...  n  H  is  desired,  and  the 
data  from  a  single  experiment  yield  X  , . . . ,X  as  tests  of  H  , . . . ,H  , 
respectively.   Combining  the  X    can  provide  a  test  of  H. 

In  Section  1.2  several  non-parametric  methods  of  combination 
are  introduced.   A  literature  review  of  comparisons  of  these  procedures 
is  given  in  Section  1.3.   The  remainder  of  this  chapter  is  primarily 
a  literature  review  of  more  specific  aspects  of  the  combination  problem. 
We  make  some  minor  extensions  which  are  identified  as  such. 


1 . 2   Non-parametric  Combination  Methods 

Suppose  that  H.  is  rejected  for  large  values  of  X    .   Define 

L    =  1  -  F  (X   J,  where  F.  is  the  cumulative  distribution  function 

of  X    under  H..   If  X    is  a  continuous  random  variable,  then  L 

is  uniformly  distributed  on  (0,1)  under  }) . .   Many  of  the  well-known 

1 

methods  of  combination  may  be  expressed  in  terms  of  the  L    .   Such 
methods  considered  here  are: 

(1)  T^^^=  -2Z£nL^^^  (Omnibus  method,  Fisher  [13]) 

(2)  T^^-*  =  -z$"^(l*-^^)  (Normal  transform,  Liptak  [26]) 

(3)  T     =  -inin  1^     (Minimum  significance  level ,  Tippet  t  [42]) 

(4)  T     =  -max  L     (Maximum  significance  level,  Wilkinson  [44]) 
.,.0')  _  ,,„r,   ,(i) 


(5)  T    =  2);£n(l  -  L^  ^]     (I'earson  [36,]) 

(6)  T^^^  =  -EL*-"-^  (Edgington  [12]). 


As  the  statistics  are  defined  here,  H  is  rejected  when  large  values  are 
observed.   Figure  1  (page  4)  shows  the  rejection  regions  for  the  sta- 
tistics defined  above  when  k  =  2. 

In  the  continuous  case,  the  null  distributions  of  these  statis- 
tics are  easily  obtained.   They  are  all  based  upon  the  fact  that  the 
L    are  uniformly  distributed  under  H.   It  is  easily  established  that 
this  is  true.   The  cumulative  distribution  function  for  L    is 

P{L  11}=   P{1  -  F(x)  <  i.) 

=  1  -  P{F(x)  <    1   -   1} 
=  1  -  P{x  <  F~''^(l  -  £)} 

=  1  -  f{f"-'-(i  -  I)} 

=  1  -  (1  -  £)  =  £. 

(N) 
That  T    has  a  normal  distribution  with  mean  0  and  variance  k 

(m)       (H) 
follows  trivially.   The  statistics  T    and  T    are  seen  to  be  based 

on  the  order  statistics  of  a  uniform  random  variable  on  (0,1)  and 

therefore  distributed  according  to  beta  distributions. 

(F)  (F) 

That  T    and  T    are  distributed  as  chi-squares  on  2k  degrees 

of  freedom  is  established  as  follows.   The  probability  density  function 

of  L^i^  is 


Let  S  =  -2J.nL.   Then 


It  follows  that 


-S/2    dL   -1   -S/2 
^  =  ^    '  ^  =  T^    ■ 


Edgington's  statistic,  T    ,  is  a  sum  of  uniform  random  variables. 
As  shown  by  Edgington,  significance  levels  can  be  established  for 


L 
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J(m) 
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Figure  1. 


L^"  I  (I) 

J(A) 

Rejection  Regions  in  Terms  of  the  Significance  Levels 
for  k  =  2. 


1    c  ^(A) 

values  ot  T    on  Lhe  basis  of  the  following  equation  [12]: 


(A)_._t^  ^^At::!)^      k(t^      u^^^  uu-s)"^ 

k!     4''   k!     S^   k!   ^•••^^S^~k!~   ^^-1^ 


P{t(^)  >-t}  =f--  (SlLli)_+  (k  lt^2)__  k  (t-3)%     k  (t-S) 

k!  ^r    k!   4''  k!   4'^  k!  +'--+(c) — n- 

where  S  is  the  smallest  integer  greater  than  t. 


1.3  A  Comparison  of  Non-parametric  Methods 

The  general  non-parametric  methods  of  combination  are  rules 

prescribing  that  H  should  be  rejected  for  certain  values  of 

A^^)   t(2)        ,(k)^    ^      ,  ^ 

L'-  >  L  ,...,  L  J.  Several  basic  theoretical  results  for  non- 
parametric  methods  of  combination  are  due  to  Birnbaum  [7],  Some  of 
these  results  are  summarized  in  the  following  paragraphs. 

Under  H^,  L    is  distributed  uniformly  on  (0,1)  in  the  con- 
tinuous case.   When  H^  is  not  true,  L^^^  is  distributed  according  to 
a  non-increasing  density  function  on  (0,1),  say  g.(L^^^),  if  X^^^ 
has  a  distribution  belonging  to  the  exponential  family.   Some  overall 
alternative  hypotheses  that  may  be  considered  are: 

H^:   One  or  more  of  the  L  ^  's  have  non-uniform  densities 

H^:   All  of  the  L    's  have  the  same  non-uniform  density  g(L). 

H^:   One  of  the  L    's  has  a  non-uniform  density  g.(L  ■^'*). 
'^\  ^^    ^^^   appropriate  alternative  hypothesis  in  most  cases  where  prior 
knowledge  of  tlic  alternative  densities  g .  (l    )  is  unavailable  (7J. 

The  following  condition  is  satisfied  by  all  of  the  metliods 
introduced  in  Section  1.2. 

Condition  1:  If  H  is  rejected  for  any  given  set  of  L  's,  then 
it  will  also  be  rejected  for  all  sets  of  L^^^^'s  such  that  L^^^*  2  L*"^'* 
for  each  i  [7] . 


It  can  be  shown  that  the  best  test  of  H  versus  any  particular 
alternative  in  H  must  satisfy  Condition  1.   It  seems  reasonable, 
therefore,  that  any  method  not  satisfying  Condition  1  can  be  elim- 
inated from  consideration  [7]. 

In  the  present  context.  Condition  1  does  little  to  restrict 
the  class  of  methods  from  which  to  choose.   In  fact,  "for  each  non- 
parametric  method  of  combination  satisfying  Condition  1,  we  can  find 
some  alternative  H   represented  by  non-increasing  functions 
g, (l   J,...,g  (l   J  against  which  that  method  of  combination  gives 
a  best  test  of  H"  [7] . 

It  should  be  noted  that  direct  comparison  of  general  combining 
methods  with  respect  to  power  is  difficult  in  typical  contexts.   The 
precise  distributions  of  the  g . (l   )  under  the  alternative  hypothesis 
are  intractible  except  in  very  special  cases. 

When  the  X    have  distributions  belonging  to  the  one-parameter 

exponential  family,  the  overall  null  hypothesis  can  be  written  H: 

g(l)  _  q(1)       (k)  _   (k)    „  .   ,.     .   u    ■      u        A 

~   0   '•••»      "   0   ■   Rejection  of  H  is  based  upon 

(X   ,...,X   J.   It  is  reasonable  to  reject  the  use  of  inadmissible 

tests.   A  test  is  inadmissible  if  there  exists  another  test  which  is 

at  least  as  powerful  for  all  alternatives  and  more  powerful  for  at 

least  one  alternative.   Birnbaum  proves  that  a  necessary  condition  for 

the  admissibility  of  a  test  is  convexity  of  the  acceptance  region  In 

the  (X    ,...,X   J  hyperplane.   For  X    with  distributions  in  the 

(p)       (M) 
exponential  family,  T     and  T     do  not  have  convex  acceptance  regions 

and  are  therefore  inadmissible  [7]. 

Although  Birnbaum  does  not  consider  Edgington's  method,  we  see 

(A) 
that  it  is  clear  that  T    must  also  be  inadmissible.   For  instance. 


for  k=2,  consider  the  points  (0,c),  (c,0),  and  (c/2,c/2)  in  the 
(l    ,L   }  plane  which  fall  on  the  boundary  of  the  acceptance  region 
T    >  c.   The  points  in  the  (x^^\x^^^3  plane  corresponding  to  (0,c) 
and  (c,0)  would  fall  on  tlie  axes  at  -  (and  --)  .   The  point  correspond- 
ing to  (c/2,c/2)  certainly  falls  interior  to  the  boundaries  described 
by  the  points  corresponding  to  (c,0)  and  (0,c).   The  acceptance  region 
can  not,  therefore,  be  convex  and  hence  T^'^^  is  inadmissible.   This 
argument  is  virtually  the  same  as  that  used  by  Birnbaum  to  establish 
the  inadmissibility  of  T    and  T   . 

For  a  given  inadmissible  test  it  is  not  known  how  to  find  a 
particular  test  which  dominates.   Birnbaum,  however,  argues  that  the 
choice  of  which  test  to  use  should  be  restricted  to  admissible  tests. 
The  choice  of  a  test  from  the  class  of  admissible  tests  is  then  contin- 
gent upon  which  test  has  more  power  against  alternatives  of  interest  [7] 

In  summary  of  Birnbaum's  observations,  since  T^'''^  and  T^^^  do 
not  in  general  form  convex  acceptance  regions  in  the  (x   ,  .  .  .  ,X^'^^) 
hyperplane,  they  are  not  in  general  admissible  and  can  be  eliminated  as 
viable  methods.   We  can  extend  Birnbaum's  reasoning  to  reach  the  same 
conclusion  about  T    .   By  inspecting  the  acceptance  regions  formed  by 
the  various  methods,  Birnbaum  also  observes  than  T  ^^  is  more  sensitive 
to  H^  (departure  from  H  by  exactly  one  parameter)  that  J^^K      The  test 

1    ,  however,  has  better  overall  sensitivity  to  H   (71. 

A 

Littell  and  Folks  have  carried  out  comparisons  of  general  non- 
parametric  methods  with  respect  to  exact  Bahadur  asymptotic  relative 
efficiency.   A  detailed  account  of  the  notion  of  Bahadur  efficiencies 
is  deferred  to  Section  2.1. 


In  ti.eir  first  investigation  [26],  Littell  and  Folks  compare 

M)      ^(N)   „(M)     ^  ^(m)    ^^ 

'■        '    ^         '  ^    '  ^^°    ^         '      The  actual  values  of  the  efficiencies  are 

given  in  Section  2.3.   The  authors  show  that  T^''^  is  superior  to  the 
other  three  procedures  according  to  this  criterion.   They  also  observe 
that  the  relative  efficiency  of  T^'"^  is  consistent  with  Birnbaum's 
observation  that  T^'"^  performs  well  versus  H  . 

Further,  Littell  and  Folks  show  that  T^^\  with  some  restric- 
tions on  the  parameter  space,  is  optimal  among  all  tests  based  on  the 
X  "■  as  long  as  the  X  ^^  are  optimal.  This  result  is  extended  in 
a  subsequent  paper  [28]  by  showing  that  T^^^  is  at  least  as  efficient 
as  any  other  combination  procedure.  The  only  condition  necessary  for 
this  extension  is  equivalent  to  Birnbaum's  Condition  1.  A  formal  state- 
ment of  this  result  is  given  in  Section  2.3. 


^•^      Parametric  Combination  Methods 

The  evidence  thus  far  points  strongly  to  T^^^  as  the  choice 
among  general  non-parametric  combination  procedures  when  prior  knowledge 
of  the  alternative  space  is  unavailable.   When  the  distributions  of  the 
X    belong  to  some  parametric  family,  or  when  the  alternative  param- 
eter space  can  be  characterized,  it  may  be  possible  that  T^^^  and  the 
other  general  non-parametric  methods  can  be  improved  upon.   A  summary 
of  such  investigations  follows. 

Oosterhoff  [3jJ  considers  the  combination  of  k  normally  dis- 
tributed random  variables  with  known  variances,  and  unknown  means 

V^,U^,...,\i^.      The  null  hypothesis  tested  isH;y   =y   =...=y=0 

1    2    *  '  *     k 

versus  the  alternative  H^:  p .  >  0,  with  strict  inequality  for  at  least 


one  i.   lie  observed  that  many  combination  problems  reduce  to  this 
situation  asymptotically.   The  difference  in  power  between  a  particular 

test  and  the  optimal  test  for  a  given  (u  ,VJ  ,  .  .  .  ,ii  )  is  called  the  short- 

(F) 
coming.   Oosterhoff  proves  that  the  shortcomings  of  T    and  the  maximum 

likelihood  test  go  to  zero  for  all  (y^  ,  .  .  .  ,1J,  )  as  the  overall  signif- 

1      k 

icance  level  tends  to  zero.   The  maximum  shortcoming  of  the  likelihood 

(F) 
ratio  test  is  shown  to  be  smaller  than  the  maximum  shortcoming  of  T 

Oosterhoff  derives  a  most  stringent  Bayes  test  with  respect  to 
a  least  favorable  prior.   According  to  numerical  comparisons  (again 
with  respect  to  shortcomings),  the  most  stringent  test  performs  sim- 
ilarly to  the  likelihood  ratio  test.   The  likelihood  ratio  test  is  much 

easier  to  implement  than  the  most  stringent  test  and  is  tiierefore  pre- 

(F) 
ferable.   Fisher's  statistic,  T   ,  is  seen  to  be  slightly  more  powerful 

than  the  likelihood  ratio  test  when  the  means  are  similar;  the  opposite 

is  true  when  the  means  are  dissimilar.   A  simple  summing  of  the  normal 

variates  performs  better  than  all  other  methods  when  the  means  are  very 

similar  [33] . 

Koziol  and  Perlman  [20]  study  the  combination  of  chi-square 

variates  X  "^   ~  x,,  (9.)-   The  hypothesis  test  considered  is  H: 

i   ^ 

6,  =  . . .  =  6,  =  0  vs  H  :  0.  >  0  (strict  inequality  for  at  least  one  i) 
1  k         A   1  -1     ^ 

where  the  9.  are  non-centrality  parameters.   The  v.  correspond  to  the 
respective  degrees  of  freedom.   An  earlier  Monte  Carlo  study  by 
Bhartacharya  [6]  also  addressed  this  problem  and  compared  the  statistics 
^(F)^  j(m)  ^    ^^^  Yy^(i)  ^      Bhattacharya  concluded  that  EX^^^  and  T^^^  were 
almost  equally  powerful  and  that  both  of  these  methods  clearly  dominated 
T    .   Koziol  and  Perlman  endeavor  to  establish  the  power  of  T    and 


10 


T.X  in  some  absolute  sense.   To  do  this,  they  compare  T  ^^  and  EX^"^^ 

to  Bayes   procedures  since  Bayes   procedures  are  admissible  and  have 
good  power  in  an  absolute  sense  [20]. 

When  the  v^  are  equal,  LX   ^      is  Bayes   with  respect  to  priors 
giving  high  probability  to  (^ j^»  •  •  •  .9;^)  central  to  the  parameter  space 
(Type  B  alternatives).   The  test E exp{yX^^^ }  is  Bayes   with  respect 
to  priors  which  assign  high  probability  to  the  extremes  of  the  param- 
eter space  (Type  C  alternatives).   For  unequal  v  's  the  Bayes   tests 

i 

have  slightly  altered  forms.   The  Bayes  procedures  are  compared  to 

t(F)  rr,(m)  (N)  ^    ,  o  ^ 

i    ,  i    ,  and  i     tor  k=Z   for  various  values  of  (v  ,v  )  via  numerical 

tabulations  and  via  the  calculation  of  power  contours. 

The  statistic,  T   ,  is  seen  to  have  better  power  than  the  other 
tests  for  Type  C  alternatives  but  performs  rather  poorly  in  other  situa- 
tions.  The  Bayes  test  performs  comparably  to  T^'"''  for  Type  C  alterna- 
tives and  is  much  more  sensitive  to  Type  B  alternatives  than  T    .   The 

.   .     (N) 
statistic,  T    ,  IS  relatively  powerful  over  only  a  small  region  at  the 

center  of  the  parameter  space.   The  statistic,  T*^^\  is  seen  to  be  dom- 
inated by  some  other  procedure  for  each  value  of  k  investigated.   The 
statistics,  T   and  EX    ,  are  good  overall  procedures,  with  T^^^  more 
sensitive  to  Type  C  alternatives  and  ZX^^"*  more  sensitive  to  Type  B 
alternatives.   For  v  >  2,  t"^^^  is  more  sensitive  to  Type  B  alternatives 
than  ZX    is  to  Type  C  alternatives  and  T^^^  is  therefore  recommended. 
The  opposite  is  true  for  v  =  1.   These  observations  were  supported  for 
k>2  through  Monte  Carlo  simulations. 

Koziol  and  Perlman  also  consider  the  maximum  shortcomings  of 
the  tests.   In  the  context  of  no  prior  information,  they  show  that  T*-^-* 
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minimizes  the  maximum  shortcomings  for  \> .    1   2   while  ZX    minimizes  the 
maximum  shortcoming  for  v   =l.   An  additional  statistic  can  be  consid- 


1 


XX)  _  ^f^(i) 


I, 


ered  when  v.  =  l.   it  is  T^^^  =  e(x^  0 '^  the  sum  of  chi's  procedures. 

(X) 
For  k-2,  T    is  powerful  only  for  a  small  region  in  the  center  of  the 

(y  ) 
parameter  space.   For  large  k,  the  performance  of  T    becomes  progres- 
sively worse.   It  can  be  said  that  T^   performs  similarly  to  T   . 


1. 5   Weighted  Methods  of  Combination 


Good  []4]  suggests  a  weighted  version  of  Fisher's  statistic, 
T    =  -ZX.2.nL    .   He  showed  that,  if  the  X.  are  all  different,  signif- 
icance probabilities  can  be  found  by  the  relationship 

?{T^^^    >   x)   =      E  A   exp(-x/A  ) 
,  r   "^     r 
r=l 


where 


A  = 


(G) 
Zelen  [45]  illustrates  the  use  of  T^    in  the  analysis  of 

incomplete  block  designs.   In  such  designs,  it  is  often  possible  to 
perform  two  independent  analyses  of  the  data.   The  usual  analysis 
(intrablock  analysis)  depends  only  on  comparisons  within  blocks.   The 
second  analysis  (interblock  analysis)  makes  use  of  the  block  totals 
only.   Zelen  defines  independent  F-ratios  corresponding  to  the  two 
types  of  analysis.   The  attained  significance  level  corresponding  to 
the  interblock  analysis  is  weighted  according  to  the  interblock  effi- 
ciency which  is  a  function  of  the  estimated  block  and  error  variances. 
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A  similar  example  is  given  by  Pape  [ 34  J  .   Pape  extends  Zelen's 
method  to  the  more  general  context  of  a  multi-way  completely  random 
design. 

Koziol  and  Perlman  [20]  also  considered  weighted  methods  for 
the  problem  of  combining  independent  chi-squares.   They  conclude  that 
wlien  prior  information  about  the  non-centrality  parameters  is  available, 
increased  power  can  be  achieved  at  the  appropriate  alternative  by  a 

weighted  version  of  the  sum  test,  Eb  X    ,  if  v   >  2  for  all  i  and  bv 

1  i  ' 

CG) 
the  weighted  Fisher  statistic,  T    ,  when  v.  1  2  for  all  i. 

1 

1.6   The  Combination  of  Dependent  Tests 

The  combinations  considered  up  to  this  point  have  been  based 
on  mutually  independent  L    arising  from  mutually  independent  statis- 
tics, X    .   As  previously  indicated,  in  such  cases,  the  functions  of 
the  L    which  comprise  the  general  methods  have  null  distributions  which 
are  easily  obtained.   l>nien  the  X^^^  (and  thus  the  L  ^^)  are  not  inde- 
pendent the  null  distributions  are  not  tractible  in  typical  cases. 

Brown  [9]  considers  a  particular  example  of  the  problem  of 

combining  dependent  statistics.   The  statistics  to  be  combined  are 

assumed  to  have  a  joint  multivariate  normal  distribution  with  known 

covariance  matrix  \    and  unknown  mean  vector  (p,  ,vi  , ,  .  .  .  ,m  )'.   The  hypoth- 

12  k 

esis   test   of   interest    is   H:      y.    =  y/    versus  H,  :    y    >     y        (strict   in- 

1    lo         1    1  -   io 

equality  for  at  least  one  i) .   A  likelihood  ratio  test  can  be  derived 
[31  J,  but  obtaining  significance  values  from  this  approach  is  difficult. 
Brown  bases  his  solution  on  T^   .   The  null  distribution  of  T^^"* 
is  not  chi-square  on  2k  degrees  of  freedom  in  this  case.   The  mean  of 
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T    is  2k  as  in  the  independent  case.   The  variance  has  covariance 
terms  which  Brown  approximates.   The  approximation  is  expressed  as 
a  function  of  the  correlations  between  the  normal  variates.   These 
first  two  moments  are  equated  to  the  first  two  moments  of  a  gamma  dis- 
tribution.  The  resultant  gamma  distribution  is  used  to  obtain  approx- 
imate significance  levels. 

1. 7   The  Combination  of  Tests  Based  on  Discrete  Data 

As  noted  in  previous  sections,  the  literature  tends  to  support 

(F) 
T    as  a  non-parametric  combining  method  in  the  general,  continuous 

data  framework.   Those  authors  who  have  addressed  the  problem  of  com- 

(F) 
bining  discrete  statistics  have  utilized  T    assuming  that  the  opti- 

mality  properties  established  in  the  continuous  case  are  applicable. 

The  problem  then  becomes  one  of  determining  significance 

(F) 
probabilities  since  T    is  no  longer  distributed  as  a  chi-square  on 

2k  degrees  of  freedom.   We  describe  the  problem  as  follows.   Suppose 

(i)*  (i)* 

L     derives  from  a  discontinuous  statistic,  X    ,  and  that  a  and  b 

(i)*  (i)* 

are  possible  values  ofL     ,Ola<b<l,  such  that  a  <  L     <b 


a. 


is  impossible.   For  a  <  C  <  b,  ?{L^'^        <  5}  =  P{L     <  a}  = 

If  L    derives  from  a  continuous  statistic,  X    ,  then  P{L    15)  =S. 

(i)A  (i) 

Since  a  <  5,  L     is  stochastically  larger  than  L    .   It  follows  that 

Fisher's  statistic  is  stochastically  smaller  in  the  discrete  case  than 

(F) 
in  the  continuous  case.   The  ultimate  result  is  that  if  T    is  com- 
pared to  a  chi-square  distribution  with  2k  degrees  of  freedom  when  the 
data  are  discrete,  the  null  hypothesis  will  be  rejected  with  too  low 
a  probability. 


14 


When  k,  tiie  number  of  statistics  to  be  combined,  is  small  and 

(n  .n^, . . . ,n  ) ,  the  numbers  of  attainable  levels  of  the  discrete  sta- 

(F) 
tistics  are  small,  the  exact  distribution  of  T    can  be  determined. 

Wallis  [43]  gives  algorithms  to  generate  null  distributions  when  all 
of  the  X    are  discrete  and  when  one  X    is  discrete.   Generating 
null  distributions  via  Wallis'  algorithms  becomes  intractible  very 
quickly  as  k  and  the  number  of  attainable  levels  of  the  X    increase. 
The  generation  of  complete  null  distributions  is  even  beyond  the  capa- 
bility of  usual  computer  storage  limitations  in  experiments  of  modest 
size.   The  significance  level  attained  by  a  particular  value  of  T 

can  be  obtained  for  virtually  any  situation  with  a  computer,  however. 

(F) 
A  transformation  of  T    which  can  be  referred  to  standard  tables  is 

indicated. 

A  method  suggested  by  Pearson  [37]  involves  the  addition,  by 

a  separate  random  experiment,  of  a  continuous  variable  to  the  original 

discrete  variable  and  thus  yielding  a  continuous  variable.   Suppose  X 

can  take  on  values  0,  1,  2,...,  n.  with  probabilities  p_  ,  p, , .  .  . ,  p   . 
n-L  1  0    1       n-j^ 

Let  pf^^  =   E   p  .   Note  that  P*^^^  <P*^^^   <  ...  <  P^""^  <  P*^"")  =  1 
J     x=j   ^  "i    ^i-1  1      0 

If  the  null  hypothesis  is  rejected  for  large  values  of  X    ,  then  the 

^ J.       >  J  =  0,1,2,  ...  ,n.  ,  are  the  observable  significance  levels  for  the 

.th 

1    test;  I.e.,  the  observable  values  of  the  random  variable 

L    =  1  -  f(x    -  ij  under  the  null  hypothesis.   Denote  by  U    , 

i  =  l,2,...,k,  mutually  independent  uniform  random  variables  on  (0,1). 

Pearson's  statistic  is  defined  as 


15 


We  now  establish  that  L J^   (x   \u   )  is  uniformly  distributed 
on  (0,1)  if  and  only  if  X  "^   and  U    are  independent  and  U    is  uni- 
formly distributed  on  (0,1).   Omitting  the  superscripts  for  convenience, 
define  the  random  variable  (X,U)  by  L   =  L(X)  -  U  P{X}  where  0  <  U  <  1. 
It  follows  that 

P{X  =  X,  U  <  u)  =  P(L(x)  -  u  P(x)  <  Lp  <  L(x)} 
=  u  P 

X 

=  P{X  =  x}   P{U  <  u}. 

The  statistic,  -2I£nL    ,  thus  has  an  exact  chi-square  distri- 

P 

bution  with  2k.  degrees  of  freedom.   Exact  significance  levels  can  be 
determined  for  any  combination  problem.   The  concept  of  randomization 
to  obtain   statistics  with  exact  distributions  has  been  debated  by 
statisticians.   That  a  decision  may  depend  on  an  extraneous  source  of 
variation  seems  to  violate  some  common  sense  principle.   Pearson  [37] 
argues,  however,  that  his  randomization  scheme  is  no  more  an  extran- 
eous source  of  variation  than  is  the  a  priori  random  assignment  of 
treatments  to  experimental  units. 

Lancaster  [21]  considers  a  pair  of  approximations  to  T   . 
Although  Lancaster  does  not  consider  Pearson's  method,  the  statistics 

he  introduces  can  be  expressed  in  terms  of  the  L     [19]: 

P 
2 
1.   Mean  chi-square  (X  ): 

m 


E(-2JlnL^^^)  = 


1       ... 

(-2£nl/^0  du 
>0       P 


2-{L^^''(X)£nL'-''\x) 
-L^^''(X+l)£nL^^^(X+l)}/P(X).      (1.3) 
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'2 
ii.   Median  chi-square  (X   ): 

m 

Median  {-ISLnL^^^    =   -2!in^[L^^\x)    +  L^^Ux+l)  }      if  L^'-^X+l)  ^   0 

=  2-2  nL^'-''(X)  if  L*''-\x+l)  =  0. 

2  2 

The  expectation  of  X   is  2 .   The  variance  of  X   is  sliehtlv  less 

ra  m  ^ 

than  4.   The  median  chi-square  is  introduced  because  of  its  ease  of 
calculation.   With  the  ready  availability  of  pocket  calculators  with 

£n  functions,  this  justification  no  longer  seems  valid.   The  expecta- 

'2  '2 

tion  of  X   is  less  than  2.   The  alternate  definition  of  X   for  when 
m  m 

L    (X+1)  =  0  is  intended  to  reduce  the  bias  (without  increasing  the 
difficulty  of  calculations)  [21]. 

David  and  Johnson  [10]  undertake  a  theoretical  investigation  of 

'2 
the  distribution  of  X   .   They  prove  that  as  n,  the  number  of  attainable 

m 

levels  of  X     (and  hence  L    ),  increases  without  limit,  the  moments  of 

'2 
X   converge  to  those  of  a  chi-square  distribution  on  2  degrees  of 
m  ° 

2 
freedom.   We  obtain  a  similar  result  for  X   by  adapting  David  and  John- 

m 

'2 
son's  proof  for  X    (superscripts  are  omitted  for  convenience).   From 

2 
the  definition  of  X   given  in  equation  (1.3),  it  follows  that 


9   L 

lira   E(X  )   =  lim  E( 

m 


n  -»-  oo  ■'Q 

P(x.)^0 
(J  =  1>2 n.) 


1  , 

(-2£nL  )du)  , 
P 


X 


Then, 


n 


9  u 

lim  E(X  )   =  lim  Z  P(x.) [ 

J=l 


1  , 

(-2JlnL  )du] 
0       P 


,       n        rl 
(-2)   lim   Z  P(x.)[    £n(L(x.)  -uP(x.))du] 
j  =  l    J   ^        J        J 


=  (-2)^  lim  ZP(x.) [ 


1 

{(L(x.)  -uP(x.)  -1) 

0     J      J 


lim  E(X^)^  =  -T  (L(x.)  "  uP(x.)-l)^  +  ...}du]'^. 
m       ^     J        J 
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(Note:       £n(a)    =    (a-1)    -  |-(a-l)^   +  j(a-l)^   -    ...) 
Upon   performing    tlie    integrations, 

0       1  I, 

limE(X   )'   =    (-2)      limZP(x.)[(L(x.)    -  -^P(x.)    -   l) 
+f([L(x    )    -   1]2   -  ip(x.)(L(x.)    -   l) 

-  3[P(x.)]^)  +  ...]^. 

rl 
Since   all   of    tlie    terms   in   the  expansion  of    [        £n(L(x.)    -  uP(x.))du]^ 

Jq  J        J 

are  multiplied  by  P(x  )  it  is  evident  that  u(P(x.))  gives  rise  only  to 

-^  J 

second-order  terms  in  P(xJ  which  can  be  ignored  in  the  limit.   The 


limit  thus  reduces  tc 


(-2)''   lim 


n       /• 

^   P(x.)[   £nL(x.)du]' 


=  (-2)^  lim  XP(x.)[£nL(x.)]^ 

(-1 
=  (-2)      [JlnL(x)]''  dL(x). 

^ 


Letting  Y  =  -  JlnL(x)  yields 


/T^b.b      -Y   b        b 
(-2)  (-1)      e   Y   dY  =  2b! 


'0 


which  gives  the  b''   moment  of  a  chi-square  distribution  with  2  degrees 
of  freedom. 

The  convergence  of  moments  does  not  in  general  imply  convergence 
in  distribution.   However,  if  there  is  at  most  one  distribution  function 
F  such  that   lim  |  x"^  dF  =  fx'^  dF,  then  F  ->  F  in  distribution  [8]. 
Since  the  chi-square  distribution  is  uniquely  defined  by  its  moments, 
it  follows  that  X^  ->  xl      in  distribution. 


ra   ^2 
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1.8   A  Preview  of  Chapters  II,  III,  and  IV 

In  Section  2.1,  the  notion  of  Bahadur  asymptotic  relative 

efficiency  is  introduced.   In  Section  2.2,  we  derive  the  Bahadur  exact 

(A)       (P) 
slopes  for  T    and  T    .   The  results  due  to  Littell  and  Folks  men- 
tioned in  Section  1.3  are  summarized  in  detail  in  Section  2.3,   In 
Section  2.4,  we  extend  the  optimality  property  for  T^^^  given  by  Littell 
and  Folks  to  the  discrete  data  case. 

Chapter  III  deals  with  a  particular  combination  problem:   the 

combination  of  mutually  independent  binomial  experiments.   Fisher's 

(F) 
method,  T    ,  is  compared  to  several  methods  which  are  based  directly 

on  the  X  "■   (rather  than  the  L^^^  .   Comparisons  are  made  via  both 
approximate  and  exact  slopes  in  Sections  3.3  and  3.4.   The  tests  are 
also  compared  by  exact  power  studies.   These  results  are  given  in 
Section  3.5.   A  summary  of  the  tests'  relative  performances  follows. 
Recommendations  as  to  the  appropriate  use  of  the  methods  are  given. 

Section  1.6  describes  some  proposed  approximations  to  the  null 
distribution  of  T^^\   In  Sectionl.7,  these  methods  are  shown  to  be 
less  reliable  than  might  be  expected.   Alternative  approaches  are  also 
i:valuated. 

In  Section  4.1,  the  combination  problem  is  approached  from  the 
standpoint  of  estimation.   Confidence  regions  based  upon  T^^^  and  T^'"^ 
are  derived.   The  remainder  of  Cl,apter  IV  introduces  future  research 
problems  which  are  related  to  the  general  combination  problem. 


CHAPTER  II 


B/UIADUR  EFFICIENCIES  OF  GENERAL  COMBINATION  METHODS 


2.1   The  Notion  of  Bahadur  Efficiency 

Due  to  the  intractibility  of  exact  distribution  theory,  it  is 
often  advantageous  to  consider  an  asymptotic  comparison  of  two  compet- 
ing test  statistics.   In  typical  cases,  the  significance  level  attained 
by  each  test  statistic  will  converge  to  zero  at  an  exponential  rate  as 
the  sample  size  increases  without  bound.   The  idea  of  Bahadur  asymp- 
totic relative  efficiency  is  to  compare  the  rates  at  which  the  attained 
significance  levels  converge  to  zero  when  the  null  hypothesis  is  not 
true.   The  test  statistic  which  yields  the  faster  rate  of  convergence 
is  deemed  superior.   A  more  detailed  definition  follows. 

Denote  by  (Y  ,Y  ,  ...)  an  infinite  sequence  of  independent 
observations  of  a  random  variable  Y,  whose  probability  distribution  ?„ 
depends  on  a  parameter  9  £0. 

Let  H  be  the  null  hypothesis  H:  ^  ^'^r,   ^nd  let  A  be  the  alterna- 
tive A:   8  eO  -  0  .   For  n  =  1,2,...,  let  X  be  a  real  valued  test 
u  n 

statistic  whicli  depends  only  on  the  first  n  observations  Y  ....,Y  . 

1       n 

Assume  that  the  probability  distribution  of  X   is  the  same  for  all 

n 

0  C0  .   Define  the  significance  level  attained  by  X  by 
^  n 

L^  =  1-F^(X^)  where  ^^M-^^i^i    x}.   Let  {X^^}  =  iX^,X^,  .  .  .  ,X^,  .  .  .} 
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20 


denote  an  infinite  sequence  of  test  statistics.   In  typical  cases,  there 

exists  a  positive  valued  function  c(6),  called  the  exact  slope  of  {X  }, 

such  that  for  6e6-6  . 

(-2/n)  In   L  ->c(e) 
n 

with  probability  one  [0].   That  is, 

P{lim  —  £n  L  =  cCG)}  =  1. 
n      n 

If  {X    }  and  {X    }  have  exact  slopes  c  (9)  and  c  (0) ,  respectively 

then  the  ratio  'i>-,2^^^    ^   c  (0)/c  (0)  is  the  exact  Bahadur  efficiency  of 

{X^^h  relative  to  {X^^^}. 
n  n 

An  alternative  interpretation  of  ({)   (9)  is  that  it  gives  the 

limiting  ratio  of  sample  sizes  required  by  the  two  test  statistics  to 

attain  equally  small  significance  levels.   That  is,  if  for  e  > 0,  N    (e) 

is  the  smallest  sample  size  such  that  L    <  e  for  all  sample  sizes 

n  ' 

n>N  ^  (£),  i  =  1,2,  then  as  e  tends  to  zero  [3] 

lira  -YTT^   =  <P,.(Q)- 

The  following  theorem  due  to  Bahadur  [3]  gives  a  method  for 
calculating  exact  slopes.   A  proof  is  given  by  I.  R.  Savage  [40]. 

Theorem  1.   Suppose  {T  }  is  a  sequence  of  test  statistics  which  satisfies 

n 

the  following  two  properties: 

1.  There  exists  a  function  b(6),  0<b(6)  <  «>,  such  that 
T  //n-+b(e)  with  probability  one  [0], 

2.  There  exists  a  function  f(t),  0<f(t)  <  oo,  which  is 
continuous  in  some  open  set  containing  the  range  of 
b(0)  such  that  for  each  t  in  the  open  set 

^£n[l-F(A't)]->f(t),  where  F  (t)=l-P  (T  <t). 
n         n  n  o   n 

Then  the  exact  slope  of  {t  }  is  c(0)  =  2f(b(9)). 
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2.2   The  Exact  Slopes  For  T^^^  and  T^^^ 


In  terras  of  the  above  discussion  the  general  combination  problem 

can  be  defined  as  follows.   There  are  k  sequences  {X   },..., {X    }  of 

"l         \ 

statistics  for  testing  H:  9  e0    For  all  sample  sizes  n  ,...,n  ,  the 

statistics  are  independently  distributed.   Let  L    be  the  level  attained 

11 . 

1 

by  X^^\    i   =  1,2,.. .,k. 
n . 

1 

Assume  that  for  each  i  =  l,...,k,  the  sequence  (X   }  has  exact 

n . 

1 

slope  c . (Q ) ;  that  is 

—  £nL^^^  -•-c.  (0)    as  n  ^  «> 
n .    n .    1  i 

witli  probability  one  [9].   Assume  also  that  the  sample  sizes  n  ,...,n 

1       k 

n. 

satisfy  n  +  . . .  +  n,  =  nk  and   lim  —  =  A  ,  i  =  1, . . . ,k.   Th 
1  k  n     i 


A   +  . . .  +  A   =  k  and 
1  k 

-—  £nL^^^  ->-A.c.(e)     as  n  ->  « 
n     n.    11 

1 

with  probability  one  [9].   As  defined  here,  n  can  be  thought  of  as  the 
avernge  sample  size  of  tlie  k  tests. 

Two  general  combining  methods  introduced  in  Section  1.2  are 

T    ,  Edgington's  additive  method,  and  T    ,  Pearson's  method.   Deriva- 

(A)  (P) 

tions  of  the  exact  slopes  of  T  ^    and  T    follow. 
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I, 
Proposition  1.   Let  T^   =  ^  ^n   Z   L^^   The  exact  slope  of 


T^^^  is  k  min  (c . (G) ) 


/n  i=l   i 


Proof:   This  proof  requires  the  above  definition  of  T    althogh 
^P,   ~  ~   n-   """^  ^  more  obvious  definition  and  is  the  form  given  in 
Section  1.2  where  the  non-parametric  methods  are  introduced.   Nothing 
is  lost,  however,  since  equivalent  statistics  yield  identical  exact 
slopes.   Proposition  1  is  proved  by  using  Theorem  1  (Bahadur-Savage). 
The  first  step  is  to  establish  b(e)  of  Part  1  of  Theorem  1.   To  accom- 
plish this,  first  suppose  that 

Ac  (9)  =  min  {A.c.CG)}. 

It  follows  that  for  all  e  > 0,  there  exists  N = N(e)  such  that  for 

n  >N, 


T-^^^i^-C.    -^^.....^. 


with  probability  one  [0].   Then,  for  n  > N, 

L^'^>-L^'^  i  =  12      k 

nj^    n^  ,  X   x,^,  .  .  .  ,K, 

with  probability  one  [9].   It  follows  that  for  n>N, 

^1    i=i   "i      ni 
with  probability  one  [9J.   Thus,  for  n>N, 

^  ^n  i/i)  .::2  ^^  ^^(i)  ^-2  ^^  ^Jl) 
n      n     n       n.    n       nj^ 

with  probability  one  [ Gj .   It  follows   that  as  n  tends  to  infinity 
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lim  —  In   L^^^  >lim  —  In  IL^^^  >  Hm  —  Hn   kL^^\ 
n  n.  n  n.  n  n. 

Ill 

with  probability  one  [6].   Hence, 

A,cre)>— ;Ln    L^^^>Ac  re), 

11      n       n .     11 

1 

with  probability  one  [0]  since,  as  n  tends  to  infinity, 

lim  -^   Jin  kL^^  =  lim  {-^  in   L^    '    -  -  S,n  k}  =  lim  -^J,nL^  '^ 
n       n  n      n     n  n     n. 

with  probability  one  [6].   Thus, 

-^^—  ^   Ac  (9) 

with  probability  one  [6].   The  choice  of  X  c  (Q)  as  the  minimum  was 
arbitrary.   Hence, 

-S—  -»-  min  A.c.(e) 

giving  b(e)  of  Part  1  of  Theorem  1.   Now,  as  n  tends  to  infinity, 

lim  —  £n[l  -  F  (v^  t) ] 

=  lira  —  )ln  P{—  £n  ZL^^^>  /n  ^ 


n       7-      n 
/n 


1 


=  lira  -^  in   P{ZL^^^  <exp  (^)  } 
i 

=  lim  ^  in    [exp  (~^)/k\)]. 


Tlie  last  equality  if  true  since  t  is  positive  and  therefore 

,-nLk,  .   , 
exp  ( — - — ;  IS  less  than  one. 
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lim  ^^—   £n[l  -  F  »^  t] 
n         n 


lim  [-^  +  -  £n  k!]  =  ^ 
2    n  2 


This  gives  f(t)  of  Part  2  of  Theorem  1.   Thus,  from  Theorem  1, 

C  re)  =  2f(b(e))  =  kmin  X.c.  (6). 
A  .11 

1 

Proposition  2.   Let  T^^^  =  —  £n[-Z  £n(l  -  L^^^)J.   The  exact 

n  /-  n. 

vn  X 

slope  of  T^  ^  is  C  (e)=kmin  A  c.(8). 
n       r       .11 

1 

The  form  of  Pearson's  statistic  given  in  this  proposition  is 
equivalent  to  the  form  given  in  Section  1.2.   The  proof  of  Proposition  2 
entails  use  of  the  Bahadur-Savage  Theorem  (Theorem  1).   The  derivation 
of  b(9)  of  Part  1  of  the  theorem  parallels  the  derivation  of  b(6)  in 
Proposition  1.   In  order  to  establish  f(t)  for  part  two  of  the  theorem, 
a  result  due  to  Killeen,  Hettmansperger ,  and  Sievers  [18]  is  required. 
They  show  that  under  broad  conditions, 

-  Zn   f^(^  t)  -  ~  ^n   P{X   >  ^^  t}  =  oil)  (2.1) 

as  n->°°  where  f  (t)  is  the  density  function  of  X 
n  r 

Proof  of  Proposition  2.   By  a  first-order  Taylor  series  approximation, 

-Z  £n(l  -  L^^))  =  Z  L^^^  +  e    \\  L^^hl 
n.        n.      n"  n.  " 


n 


1         1 


where  e^ -^  0  as  L^""  ->  0  for  all  i.   It  follows  that  as  n  tends  to  infinity, 
i 
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T 

lim  -^  =   lim  —  JLn[-Z    Jln(i   -  L*-^^)l 

=   lim  —  SLn[i:   L^^^l 
n  n . 

1 


min    A    c    (9  )  . 
i    i 

1 


The  last  step  is  shown  in  Proposition  1.   Now,  to  derive  f(t)  of 
Part  2  of  Theorem  1  via  equation  (2.1),  note  that  the  U    =1-L 


n .       n . 
1        1 


are  mutually  independent  uniform  random  variables  on  (0,1).   Letting 

V^^^  =  -Zn   \]^^]    it  follows  that 
n.         n. 
1  1 

f  ,.s  (V)  =  e~  ,    0  <  V  <oo  . 
yd) 

n. 

1 

Letting  W^=EV^^^  =  -Lin   U^^\    it  follows  directly  that 
i  i 

c         r    \        1    k-1   -w 

n 

since  V   is  a  gamma  with  parameters  (1,1).   Now,  letting 

Y   =  -  i,n  W  =  -  £n[-Z  in   U*"^^], 
n         n  n.   ' 

1 

^Y  ^''^  "  '^~   exp{-ky  -  e~^}  .  (2.2) 

n      A 

It  follows  that,  as  n  tends  to  infinity 

lim  —  £n  P{T^^^  >  v4t"  t} 
n        n 

—  1         7 
=  lim  —  £n  P(- —  Y  >   /n    t] 
n       ^  n 
vn 


26 


=  lim  —  Hn   P{Y   >  ^} 
n  n    2 

=  lim  ^  ?.n  f^   (^)  (2.3) 

n 

by  the  result  given  in  equation  (2.1).   Substituting  (2.2)  into  (2,3), 

we  find  that 

lim  —  £n[ —  exp{-k(-^)  -  exp(-T-)}] 
/k         2         2 

.   -1  f-knt       ^-nt, 

=  lim  —  [— r- exp  (-—-)] 

n    z         2 

,kt    1     /-nt, ,    tk 
=  Ixm  [—  +  -  exp(  J-)]  =  ^  . 


Applying  the  result  of  Theorem  1, 


C„(9)  =  2f(b(6))  =  k  min  A.c.(e) 
P  .11 

1 


2. 3   Further  Results  on  Bahadur  Efficiencies 

Bahadur  exact  slopes  are  derived  in  Littell  and  Folks  [26]  for 
other  general  combining  methods.   Their  results  are  summarized  below. 

Test  Exact  Slope 

T*-^^       c  (6)  =  z  A.c.(e) 
r  11 


T^^^       c  (Q)  =hHKcAQ)S]^ 


(M) 
T   ^       c.^(O)  =  k  min  A.c.(O) 

L 

T*''"''       C  (0)   =  max  X  c  (0 ) 
m         .    i  i 

1 


Thus  T    and  T    have  the  same  exact  slope  as  T    .   The  relationship 
among  these  quantities  is  displayed  in  Figure  2. 
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X|C|=(l  +  y2)  X2C2 


x,  c 


Note:  I  and  JK:  Cm<C;^<c^  <Cp 
H  and  ^:  c^<c^<c^j  <  Cp 
HI  and  rZ;     c^<c^<c^  <  Cp 


Figure  2.   Relative  Sizes  of  Exact  Slopes  for  k = 2 . 
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(F) 
The  optimality  property  of  T    established  by  Littell  and 

Folks  [28]  is  mentioned  in  Section  1.3.   The  detailed  result  is  given 
here  as  a  theorem. 

Theorem  2.   Let  T   be  any  function  of  T    ,...,T    which  is 

non-decreasing  in  each  of  the  T    ,  that  is,  t   <t' t   <t' 

n^  1    1     '  k  "  k 

implies  T^^^^  '  ■  •  •  .  t^^)  <  T^^'^J  '  •  •  •  .  t^^)  .   Then  the  exact  slope  c(6)  of 

T   satisfies  c(9)  <  Z  A  c  (9). 
n  ~    1  i 

The  non-decreasing  condition  is  equivalent  to  Birnbaum's 
Condition  1.   (See  Section  1.3.)   The  condition  is  not  very  restric- 
tive; it  is  satisfied  by  any  method  thus  far  introduced  and  by  virtually 
all  other  reasonable  statistics. 


(F) 
2-4   Optimality  of  T    in  the  Discrete  Data  Case 


Littell  and  Folks  established  that  the  exact  slope  for  T 


(F) 


IS  Cp(9)  -  Z  ^^c^(9).    This  derivation  is  contingent  on  the  fact  that 
1     has  a  chi-square  distribution  on  2k  degrees  of  freedom.   This  is 
true  only  when  the  X  ^   are  distributed  according  to  continuous  distri- 
butions.  A  proof  that  the  exact  slope  of  T*-^-"  is  Z  A  c  (9)  „hen  the  X^^^ 

i  i 

are  discrete  follows. 

Suppose  X^^   can  take  on  values  0,  1,  2,...,n.  with  probabil- 
i  ^ 

n, 

ities  p    p    ...,p   .   Let  P^^^=   Z   p  .   Note  that 
n.   ^  ^n.-l  ^  •  •  •  ^  Pj^   <  Pq   =1.   If  the  null  hypothesis  is  rejected 
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for  large  values  of  X    ,  then  the  p/  ,   j  =  0,1,2, ...,n.  are  the 

i  -^  "'" 

observable  significance  levels  for  the  i    test;  that  is,  the  observ- 
able values  of  random  variable  L    =  1  -  F(X    -  1)  under  the  null 

n .  n. 

1  X 

hypothesis.   Assume  that  an  exact  slope  exists  for  all  tests;  that  is, 


assume  that  there  exist  functions  c,(9),  c^ (9 ) , . . . ,c,  (9)  such  that 

12  k. 

0  /  '  \ 

^—  In   L  ^  c.(0)      as      n      -I-  °° 

n .  n.  1  1 

1  1 

with   probability   one    [9]    for   i   =   l,2,...,k. 

Proposition    3.      Let   T^'^^    =    (-2   E    In   L        )'^.      If 

n  n 

1 

lim     —  Jcnfl   -    F         ( /n    t)]      exists    then    the   exact    slope   of   T  is 

n         n  n 

n  ->•  oo 

k 

c.(0)  =   E  A  c  (0). 

d      i=l  1  1 

Proof:   This  proof  utilizes  the  Bahadur-Savage  Theorem  (Theorem  1). 
To  establish  the  first  part  of  Theorem  1,  observe  tliat 

^(d)    ^  2£nL^^^         2)lnl/'^^  % 
n      /        1  k  1 


^ 


n  n 


->■    (A,c,  (9)  +  ...  +  A  c,  (9))  ^  as  n  ^-  " 
i  i  k  k 

with  probability  one  [9].   Consistent  with  the  notation  of  Theorem  1, 

denote  this  limiting  quantity  b,(9). 

d 

Now,  to  establish  f(t)  of  Part  2  of  Theorem  1,  choose 
Jl .  c  (0,1),   i  =  1,2,. ..,k.   For  each  i,  there  exists  a  j  such  that 

1    J     J-1 
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Now,  since  L^    is  a  discrete  random  variable, 
i 

n    1       n-     j      i 

1  1    "^ 

Thus 

p{L^^^  <  £.}  <  pCu^"-^  <  a  } 

n.      1  i 

where  the  U    are  mutually  independent  uniform  random  variables  on 
(0,1).   It  follows  that 

P{-2£nL*''--'  >Z\}    <   ?{-2!ln   V^^^    >   £'}. 
n^    1  ±' ' 


and  hence  that 


k       ...  k 

P{  L     -2£nL^^^  >£"}  <  P{  E  -2£n  U^^^  >£"}, 
i=l      "i  i=l 


Thus, 

^  Or.   Drrv  _  on_,  (i)i% 


-   £n  P{[E  -  2£nL_^^^]'^>A  t}  >  ^  £n  P{[E-2£n  U^^)]^>v^  t}.   (2.4) 
i 

The  quantity  Z^^  =  [E  -  2£n  U^^"*]^  is  distributed  as  the  square  root  of 


ty 


a  chi-square  with  2k  degrees  of  freedom.   It  follows  that  the  densi 

of  Z   is 
n 

f    (7)    =     1      72k-l   -Z^/2 

n       2*'  ^  r(k) 

Thus,  from  the  result  given  in  (2.1),  the  limit  of  the  right-hand  side 

of  (2.4)  can  be  written  as 

lim  — £n    f^    (/n    t) 

n  Z 

n  -><»  n 

=      lim     ^  £n    [^^^-^i (/n    t)^''^-^   exp(4   (/n    t)^] 

n->co  2  r(k)  -^ 

=      lim     ~    [(2k-l)    £n(/n    t)    -  ^  nt^] 

=     i   t^ 
2        • 
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Hence,    it    follows    from    (2.4)    tliat 

f,(t)    >-\t\ 

since  it  is  assumed  that  the  limit  of  the  left-hand  side  exists, 

Applying  the  result  of  Theoiera  1, 

0,(6)  >  ZA.c.(0). 
d        11 


By  Theorem  2, 


Hence, 


0,(9)  <  EA.c.(e) 
d        11 


0,(9)  =  ZA.o.(0). 
d         11 

(F) 
That  is,  the  exact  slope  of  T^    is  ZA.o.(9)  regardless  of  whether  the 

v(i) 

A    are  continuous  or  discrete.   The  condition  imposed  in  Proposition  3 

that   lim  —  £n[l-F    (/n  t)  ]  exist  is  not  very  restrictive.   It  is 

n  ->  CO  '^ 

satisfied  in  most  typical  cases  and  in  every  example  considered  in  this 
dissertation. 


CHAPTER  III 


THE  COMBINATION  OF  BINOMIAL  EXPERIMENTS 


3.1   Introducti 


on 


Chapter  III  deals  with  the  combination  of  binomial  experiments. 

That  is,  suppose  k  binomial  experiments  are  performed.   Let  n  ,n  ,  .  .  .  ,n 

and  X^   ,  X^   '•••'^1    be  the  sizes  and  the  observed  number?  of  suc- 
12        k 

cesses,  respectively,  for  the  experiments.  Denote  the  unknown  success 
probabilities  as  p^, p^, . . . ,P|^.  Suppose  one  wishes  to  test  the  overall 
null  hypothesis  H:  p^  =  p^^,  p^  =  p^^,  •  •  ' '  Pj,  =  P^q  ^*^^^"^  the  alter- 
native hypothesis  H^:  p^  >  p^^,  ...,  p^  >  p^^^  (with  strict  inequality 

for  at  least  one  p.).   The  problem,  then,  is  to  choose  the  best  function 

of  (Y^^^  v^'^)^  (T     ,  .   , 

n   '  ■•■'   n   ^  *°^  this  hypothesis  test. 
1  \ 

The  results  of  Chapters  1   and  II   support  T*^^^  as  a  non- 
parametric  method  with  good  overall  power  when  there  is  no  prior  infor- 
mation concerning  the  unknown  parameters.   The  method  based  on  the 
minimum  significance  level,  T^"'\  is  sensitive  to  situations  where 
exactly  one  of  the  individual  hypotheses  is  rejected.   That  is,  T^'"^  is 
powerful  versus  the  alternative  11^  of  Section  1.3. 

The  investigations  of  Koziol  and  Perlman  [20]  and  Oosterhoff  [33] 
show  that  the  general  non-parametric  combining  methods  can  be  improved 
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on  for  certain  parametric  combining  problems.   It  follows  tliat  there 

may  be  combination  methods  based  directlyon(X    ,...,X    )  that  are 

"l       \ 
superior  to  Fisher's  omnibus  procedure. 

(F) 
Chapter  III  is  a  detailed  comparison  of  T    and  several 

parametric  combination  methods. 


3. 2   Parametric  Combination  Methods 

As  stated  in  Section  1.3,  no  method  of  combination  is  most 
powerful  versus  all  possible  alternative  hypotheses.   There  are,  however, 
certain  restricted  alternative  hypotheses  against  which  most  powerful 
tests  do  exist. 

Let  the  likelihood  function  for  the  i    binomial  experiment 

be  denoted  by 

n.     ,^(i)       n  -X^^^ 
L(P,)  =  (  (,))P^    (1-  P.)        .  (3.1) 

According  to  the  Neyman-Pearson  Lemma,  if  a  most  powerful  test  of  the 

null  hypothesis  H:  p.  =  p   ,  all  i,  versus  the  alternative  hypothesis 

H  :  p.  >  p.   (with  strict  inequality  for  at  least  one  i)  exists,  it  is 
A    1     lO  ' 

to  reject  H  if 

k  L(p   ) 

i-i  '■'-i* 

Upon  substituting  (3.1)  and  taking  logs,  an  equivalent  form  of  the  test 
is  to  reject  H  if 

ZX^^^  ^n{p^(l  -  Pio)/Pio(l-Pi)^  "   ^-  (^-2) 

It  follows  that  rejecting  H  when 

T.   X^^^  >  C  (3.3) 

is  most  powerful  if  p.(l  -  p   )/p   (1-p.)  is  constant  on  i. 
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The  problem  of  combining  2x2  contingency  tables  is  closely 
related  to  the  problem  being  considered.   The  purpose  of  each  2x2  table 
can  be  interpreted  as  testing  for  the  equality  of  success  probabilities 
between  a  standard  treatment  and  an  experimental  treatment.   The  overall 
null  hypothesis  is  that  the  experimental  and  standard  success  probabil- 
ities are  equal  in  all  experiments.   The  overall  alternative  hypothesis 
is  that  the  experimental  success  probability  is  superior  in  at  least 
one  experiment. 

Cochran  [iOj  and  Mantel  and  Haenszel  [29]  suggest  the  statistic 


^   w^d^//Ew_j^p^q_^  (3.4) 


where 


w.  =  n.^n.^/Cn.^  +  n.^) ,   d.  =  p.^  -  p.^ 

for  combining  2x2  tables.   Mantel  and  Pasternak  [34]  have  discussed 
this  statistic  in  the  context  of  combining  binomial  experiments.   Each 
individual  binomial  experiment  is  similar  to  an  experiment  resulting 
in  a  2x2  table,  two  cells  of  which  are  empty  because  the  control 
success  probability  is  considered  known  and  need  not  be  estimated  from 
the  data.   The  statistic  defined  by  (3.4)  will  be  denoted  by  t^^^\ 
It  can  easily  be  shown  that  the  test  T^^^^^^  >  c  is  equivalent  to  the 
cesc  LA  >  c,  thus  T      is  the  most  powerful  test  when 

Pj^(l  ~  '^iO^'^'^iO^"'"  ~  P--^  ■'■^  constant  on  i. 

In  many  practical  combination  problems  with  binomial  experiments, 
PiO  "  ■^^^    ^°^  ^^^  ^-   "T'""^  ""Jll  hypothesis  is  then  H:  p.  =1/2,  i  =  l,2,...,k 
and  the  general  alternative  hypothesis  is  H^:  p.  >  1/2  (strict  inequal- 
ity for  at  least  one  i) .   This  is  the  hypothesis  testing  problem  under 


consideration  throughout  the  remainder  of  Chapter  III.   For  p.  =1/2, 
all  i,  T      is  uniformly  most  powerful  for  testing  H:  p   =1/2,  all  i, 
versus  H^:  Pj^  =  P2  "  P3  "  •••  "  P^.  '^  ^^'^• 

For  the  hypothesis  test  just  described,  T      can  be  toritten 

k    ,  k      1 

J:  (X^^^  -  ^  n.)/(  E  i  n.)^.  (3.5) 

1=1  J=l 

This  variate  is  asymptotically  standard  normal.   It  is  of  note  that 
this  form  is  standardized  by  a  pooled  estimate  of  the  standard  devia- 
tion.  An  alternative  statistic  can  be  formed  by  standardizing  each 
X    yielding 

T    =  —  Zi(X    -  -  n.)/(-  n.)  } 
^  2   1   4   1 

which  also  has  an  asymptotic  standard  normal  distribution.   The  statistic, 

(X) 
T    ,  is  analogous  to  the  sum  of  chi' s  procedure  which  has  been  recommended 

(x) 
for  combining  2x2  tables.   The  statistic  T  '^  is  not  in  general  equiva- 
lent to  T     ;  in  fact,  the  test  T    >  c  is  equivalent  to  the  test 

S   n^  X    >  c.   When  the  n.  are  all  equal,  T^^  and  T      are  equivalent. 
i=l  ^ 

Weighted  sums  of  the  X  ^  ,  say  S(g)  =  Z  g.  X    ,  form  a  general 
class  of  statistics.   Oosterhoff  considers  this  class  and  makes  the  fol- 
lowing observations  concerning  their  relationship  with  the  individual 
sample  sizes  [32].   It  follows  from  (3.2)  that  if  i2.n(p . /1-p . )  =  a  g  , 

tlien  the  most  powerful  test  of  H  versus  H   is  L  g   X    >c. 

A       i 

Let  p.  =—  +  e..   It  follows  that 
1    2    1 
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«.n(p./l 


p.)  =  «.n(l  +  2t-./l  -  2t.  ) 
1  1        i 


2{2e.  +  — ~ —  + 


3! 


(2£.) 

1 

5! 


+  ...} 


i    e.^-0   i' 

1 

This  implies  that  for  alternatives  close  to  the  null  hypothesis,  H, 

S(g)  is  most  powerful  if  e.  =  a  g . ;  that  is,  if  the  deviations  from  the 

null  values  of  the  p^  are  proportional  to  the  respective  g..   The  sum 

of  Chi' s  procedure,  T^^\    is  a  special  case  where  g   =  (n  )~^   It  fol- 

i     i 

lows  that  the  alternatives  against  which  t'^^^  is  powerful  is  strongly 

related  to  the  sample  sizes,  n  ,n  ,...,n  . 

12      k 

The  weighted  sum,  S(g),  may  be  a  viable  statistic  if  prior 
information  concerning  the  p.  is  available.   Under  the  null  hypothesis, 
S(g)  is  a  linear  combination  of  binomial  random  variables,  each  with 
success  probability  1/2.   The  null  distribution  of  S (g)  will  therefore 
be  asymptotically  normal.   The  proper  normalization  of  S(g)  is  analogous 
to  that  of  1^^^'"^  given  in  (3.5). 

A  well-known  generalization  of  the  likelihood  ratio  test  is  to 
reject  the  null  hypothesis  for  large  values  of  -2  £n{  sup  L(9,x)/sup  L(e,X)}. 
It  is  easily  shown  that  for  the  hypothesis  test  being  considered,  the 
likelihood  ratio  statistic  is 


where 


n.    (n.  -X^^^£n(l 


v(i)     y(i)   , 

n .      n.    2 
1       1 


r 


I{- 


,(i) 


1  if 


-^>  1/2}  =  J 


X 


n. 

1 

(1) 


>  1/2 


0   if 


<  1/2 
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Under  broad  conditions,  wliich  are  satisfied  in  this  instance,  the 

(JD) 

Statistic  T     has  an  asymptotic  chi-square  distribution  with  k degrees 

of  freedom. 

Suppose  z.,  i  =  l,2,...,k  are  normal  random  variables  with 

means  u.  and  variance  1.   The  likelihood  ratio  test  for  H:  M.  =0, 

i  =  l,...,k  versis  H  :  u.  >  0  (with  strict  inequality  for  at  least  one  i) 

is  to  reject  H  for  large  values  of 

k  2 

Y.    z.  Kz.  >  0}.  (3.6) 

1=1  ^     ^ 

For  the  binomial  problem,  an  "approximate  likelihood  ratio"  test  is  then 

to  reject  for  large  values  of 

,(ALR)  .  ^    (1)  _1   ^2  1     „^(1)  ^1 

.  1        2   1   A   1        -  2   1 
1=1 

1      1    >' 
since  (X  -—  n . ) /  (—  n.)^  is  asymptotically  a  standard  normal  random  var- 
1   2   1   4   1 

iable  under  H.   The  exact  null  distribution  of  (3.6)  is  easily  derived. 
Critical  values  are  tabled  in  Oosterhoff.   When  p  =  1/2,  the  normal 
approximation  to  the  binomial  is  considered  satisfactory  for  even 
fairly  small  sample  sizes.   It  follows  that  the  exact  null  distribution 

of  (3.6)  should  serve  as  an  adequate  approximation  to  the  null  distribu- 

^  ,^(ALR) 
tion  of  T     . 

3. 3   Exact  Slopes  of  Parametric  Metliods 

in  this  section,  the  exact  slopes  of  T    ,  T      ,  and  T 

are  compared.   We  have  not  been  successful  in  deriving  the  exact  slope 

(X) 
for  T    .   A  more  complete  comparison  of  methods  is  given  in  Section  3.4 

with  respect  to  approximate  slopes. 
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Suppose  X    is  a  binomial  random  variable  based  on  n   observa- 
n  i 

tions  with  unknown  success  probability  p..   Consider  testing  the  single 
null  hypothesis  H:  p.  =  1/2  versus  the  single  alternative  hypothesis 
H^:  p.  >  1/2. 

Proposition  A.   Let  T    =  X    .   The  exact  slope  of  T  ^^  is 

c^(9)  =  2{p^  Jin  2p^  +  (1  -  p.)  In    2(1  -  p.)}. 

Theorem  1  is  used  to  prove  Proposition  4.   There  are  several 
means  by  which  the  function  f(t)  of  Part  2  of  Theorem  1  can  be  obtained. 
Perhaps  the  most  straightforward  way  is  by  using  Chernoff's  Theorem  [1]. 
Bahadur,  in  fact,  suggests  that  the  derivation  of  f(t)  provides  a  good 
exercise  in  the  application  of  Chernoff's  Theorem. 

Theorem  3.   (Chernoff's  Theorem).   Let  y  be  a  real  valued  random 
variable.   Let  <}'(t)  =  E(e   )  be  the  moment  generating  function  of  y. 
Then,  0  <  <1>  (t)  <  «  for  each  t  and  <!>  (0)  =  1.   Let  P  =  inf{4i(t):  t  >  0}. 
Let  y^,  y^,...,  denote  a  sequence  of  independent  replicates  of  y  and 
for  n  =  1,2 ,  let  P   =  P{y-  +  ...  +  y   >  q}.   Then 

—  Jin  P  ->  £n  p    as   n  ->  °°. 
n     n 

Proof  of  Proposition  4.   For  Part  1  of  Theorem  1, 

,(i)    ,(i) 

n  .      n 
1       1 

=  - — -  ->  p 

n.     '  1 


em 


n.      1 


with  probability  one  [6]  giving  b(6).   For  the  binomial  probl 

9  =  (Pj^,P2,...,Pj^)'. 


Now,  as  n  tends  to  infinity, 


lim  —  £n(l  -F  (/^   a)) 
n .         n   1 
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=  lim  —  in   Pix'^^V^T  >  /iTT  a} 


n .     1  -    1 

1 


=  lim  —  In   P{X^^^  >  n  a} 
n.       n .      1 
1        1 


lim  —  in   P{X^^^  -  n.a  >  q} 
n .       n .     1   ~ 

1        1 


(3.7) 


The  random  variable  X    -  n.a  can  be  expressed  as 


.(i) 


\/    =  (y^L  -  a)  +  (y^  -  a)  +  ...  (y^  -  a) 


where  the  y^  are  independent  replicates  of  a  Bernoulli  random  variable 
y  witli  parameter  1/2.  Therefore,  4>(t)  of  Ctiernoff's  Theorem  is 


<!>„  .^(t)  =  I  e~^^l  +  e') 
i 


It  follows  that 


.  iTfl   ~at,     t, 
P  =  inf{-  e    (i  +  e  ):   t  >  0}, 

1    ""3.  t         t 

The  quantity  -  e    (1  +  e  )  is  minimized  for 


t  =  £n 


1-a 


Thus, 


and 


,,  =  i  -aUn  a/l-aj       [Hn   a/l-a], 
p    ^  e  (1+  e  ) 


£n  P  =  -a  in   a/l-a  +  In   -(1  +  a/l-a) 
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iiencc, 

lim  ^  in   P{X^^^    >   n^a}  =a    «,n(a/l-a)  -  Jin(-|(l  +  a/l-a) )  (3.8) 

i  i 

giving  f(a)  of  Part  2  of  Theorem  1.   Thus, 

c^(J)  =  2{p^  an   p^/l-p.  -£n  |(1  +  p^/l-p^)} 

=  2{p.  Zn   2p.  +  (1  -  p.)  Jin  2(1  -  p.)}  . 

1         1  1  X 


Following  the  notation  of  Section  2.2,  suppose  k  binomial 

experiments  are  to  be  combined  and  the  sample  sizes  n-,n„,...,n 

12      k 

satisfy  n   +  n  +  . . .  +  n   =  nk  and 

n . 
1 im  —  =X.,    i=l,...,k. 
n  -^  0° 

Then  >^,  +  .  .  .  +  A   =  k  and 
1  k 

—  £n  L*-^^  -^-X.c  (6)     as   n  ^  ». 
n     n.    11 

1 

According  to  Proposition  3,  c  (0 )  =  E  X.c  (6)  in  both  the  continuous  and 

F         1  i 

discrete  case  if  lim  —  £n[l  -  F^  U^ni)]   exists.   The  existence  of  this 

n         n      '^ 

limit  for  a  single  binomial  experiment  is  shown  in  (3.8)  of  the  proof 
of  Proposition  A.   Therefore,  for  the  binomial  combination  problem,  the 
exact  slope  for  Fisher's  method,  T    ,  is 


c„(^)  =  T.   A. {p.  £n  2p.  +  (1  -  pJJln  2(1  -  p.)}. 
r        ._   1   1        1  ^i       ^      ^i^ 

A  property  of  likelihood  ratio  test  statistics  is  that  they 
achieve  the  maximum  possible  exact  slope  [2].   Theorem  2  states  that 
the  exact  slope  for  the  combination  problem  is  bounded  above  by 
^  ^±^±^^^'      Proposition  3  shows  that  T^    achieves  this.   If  follows 
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(F) 
that  T    and  the  maximum  likelihood  test  have  the  same  exact  slope; 

that  is, 

This  relationship  is  true  regardless  of  whether  the  data  are  discrete 

or  continuous. 

Let   T  '''"   = T.   X^^  .   This  form  of  the  Cochran-Mantel- 

n        /-  ,  n. 

vn  k     1 

Haenszel  statistic  is  equivalent  to  those  previously  given  in  (3.3) 
and  (3.5). 

Proposition  5.   The  exact  slope  of  T      is 

n 

c^Ka.('^)  =  2k{p  £n  2p  +  (1-p)  J^n  2(l-p)}  where  p  =  f  Z  A.p.. 
Proof.   To  get  b(0)  of  Part  1  of  Theorem  1, 

-^ =  Z  X^^Vnk 

,    n .    n. 

k   n .    n 

1 

-  I  Z   X .p.  =  p 

with  probability  one  [9].   Now,  for  Part  2  of  Theorem  1,  as  n  tends  to 
infinity, 

lim  —  £n[l  -  F(/n  a) ] 
n 

=  lim  —  £n  P{— —  i:  X^^^  ^  >^  a} 
n        r-  ,     n . 
/n  k     1 

=  lim  k  ^  in   P{E  X*"^^  5  n  k  a}  =  f(a).  (3.9) 

nk         n . 

1 


42 


Under  the  null  hypothesis,  EX    is  a  binomial  random  variable  based 

i 
on  n  +  ...  +  n   =  nk  trials  with  success  probability  1/2.   The  quantity 

(3.9)  is  the  same  as  the  quantity  (3.7)  except  that  n.  has  been  replaced 

by  nk.   Theorem  3  can  be  directly  applied  to  line  (3.9),  yielding 

lira  —  Jin    [1   -   F(^   a)] 
n 

=   k{a   In  -^ £n  ^(1  +  a/l-a) }    =    f(a) 

1-a  2 

and  therefore 

0^,^(9)    =   2f(b(Q)) 

=   2k{p   £n(p/l-p)    -   Ini^a  +  pl-p) } 

=   2k{p    Jin   2p  +    (1-p)    Hn   2(l-p)}. 

^  ^(CMH)    ,   .      ^(F)    ,  ^(LR)   .  , 
A  comparison  of  T      relative  to  T    and  F     with  respect 

to  exact  slopes  is  given  in  the  next  section. 

Derivation  of  the  exact  slope  of  the  sum  of  chi's  procedure, 

(X) 
T    ,  has  not  been  accomplished.   An  incomplete  approach  to  the  problem 


follows . 


(y)        1  ^      (i) 

Let   t'^^    =  -  Z   nf  X        .      To   derive   b(d)    of   Part    1   of    Theorem   1, 
n  n  1      n.  ' 

1 

r^^^  n    ^    ^^'^ 

T  f^-  \'i        n . 


^ 


n   ;  n 


% 


-+ZA.P.         as      n->o° 
11 

with  probability  one  [0].   Now,  as  n  tends  to  infinity, 

lim  ^-  an   P{1  -  F(i^  a)} 


n    \n  n,         \  n  /    n, 

^1  ^     k 
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Tlie  left-hand  side  of  the  above  probability  statement  is  a  weighted  sum 
of  independent  binomial  random  variables  based  on  varying  sample  sizes 
n  ,n  , ...,n   each  with  success  probability  1/2.   The  moment  generating 
function  of  this  random  variable  is  therefore 


4-    ( 


ffi .  1  i-i) 


2^ 


n  /n 


14^ 


itI'^V'r/" 


(3.10) 


From  the  form  of  the  moment  generating  function  given  in  (3.10),  it  is 
apparent  that  the  random  variable  in  question  can  be  regarded  as  a  <jum 
n  independent  identically  distributed  variates  each  with  moment  generat- 
ing function 


n..% 


1^1    uy    ^^"/"       ,-.-    . 

2+ie  ...   -  +  -e 


.  .  .  Q) 


n,  /n 
k 


Then,  since  as  n  tends  to  infinity 

,i,zip{(^]%(l)^...  W\]"^,(k)  , 
n    V  n;    n,  V  n  7    n,   " 


na } 


n   /n.s% 


n    .    ^    K  n  J         n. 


a   >  0}, 


(t)  of  Theorem  3  is 


nt)  =  e  ^'[(^  +  f  e  1  ) 


( 2  +  i  e    )  ] 


and  p  =  inf{(J>(t)  :  t  >  0} 

The  quantity  p  has  not  been  found. 
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3.4   Approximate  Slopes  of  Parametric  Methods 

Exact  slopes  are  defined  in  Section  2.1.   In  Section  3.3  some 

comparisons  among  methods  are  made  with  respect  to  exact  slopes  and 

corresponding  efficiencies.   Bahadur  also  defines  a  quantity  called  the 

approximate  slope  [3].   Suppose  that  X  has  an  asymptotic  null  distri- 

n 

bution  F;  that  is, 

lim   F  (x)  =  F(x) 

n 
n  ->■  0° 


for  all  X.   For  each  n,  let 

L    -  1  -  F(x  ; 
n  n 

be  the  approximate  level  attained.   (Consistent  with  Bahadur's  notation, 

(a) 
the  superscript  a  stands  for  approximate.)   If  there  exists  a  c    (9) 

such  that 

^  in   L^^)  -.   c^^^O) 
n      n 

(a) 
with  probability  one  [^ ],  then  c    (9 )  is  called  the  approximate  slope 


of  {X  }, 


If  c.   (9)  is  the  approximate  slope  of  a  sequence  {x    }, 
1  n 


i  -  1,2,  then  c^  (Q)/c^      (9)  is  known  as  the  approximate  asymptotic 

efficiency  of  (x^  M  relative  to  {x    }. 
n  n 

A  result  similar  to  Theorem  1  is  given  by  Bahadur  [3]  for  the 

calculation  of  approximate  slopes.    Suppose  that  there  exists  a 

function  b(9),  0  <  b  (0 )  <  oo^  such  that 

T 

—  -.b(9) 
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with  probability  one  [0].   Suppose  that  for  some  a,  0  <  a  <  «,  the 
limiting  null  distribution  F  satisfies 

£n[l  -  F(t)]  ~  -  I  at^   as   t  -v  », 

(a)  2 

Then  the  approximate  slope  is  c    (0)  =  a[b(0)]  ,   This  result  is 

-applicable  with  a  =  1  for  statistics  with  asymptotic  standard  normal 

distributions  [3].   This  result  can  be  shown  directly  by  applying  the 

result  of  Killeen  et  ai.  given  in  (2.1). 

(a) 
The  approximate  slope,  c    (G) ,  and  the  exact  slope,  c(9), 

of  a  sequence  of  test  statistics  are  guaranteed  to  be  in  agreement  only 
for  alternative  hypotheses  close  to  the  null  hypothesis.   Otherwise, 
they  may  result  in  very  different  quantities.   One  notable  exception  is 
the  likelihood  ratio  statistic.   When  the  asymptotic  null  distribution 
is  taken  to  be  the  chi-square  distribution  from  the  well-known  -2  £n 
(likelihood  ratio  statistic)  approximation,  the  approximate  slope  of  the 
likelihood  ratio  statistic  is  the  same  as  the  exact  slope.   The  approx- 
imate slope  is  based  upon  the  asymptotic  distribution  of  the  statistic. 
Equivalent  test  statistics  may  have  different  asymptotic  null  distribu- 
tions giving  rise  to  different  approximate  slopes.   Tliis  apparent  short- 
coming does  not  exist  with  exact  slopes. 

In  typical  applied  situations,  the  significance  levels  attained 
by  T      and  T    will  be  ascertained  by  appealing  to   their  asymptotic 
normal  distributions.   Similarly,  T^^*^^  will  be  compared  to  the  appropri- 
ate chi-square  distribution  and  approximate  levels  for  t^^^"^^  will  be 
obtained  from  the  asymptotic  distribution  given  in  Section  1.2.   Approx- 
imate slopes  based  upon  these  asymptotic  distributions  would  therefore 
seem  to  afford  a  more  appropriate  comparison  of  the  methods.   In  other 
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words,  it  is  appealing  to  consider  the  null  distribution  that  will  be 
used  to  obtain  significance  levels  in  practice  when  comparing  the 
statistics.   The  only  statistic  wiiich  will  not  usually  be  compared  to 

asymptotic  distributions  is  perhaps  T      .   The  null  distribution  of 

(  PMH  ^ 
T      is  binomial  based  on  n^  +  . . .  +  n,  trials  with  success  probabil- 

1  k  ^ 

ity  1/2.    However,  even  with  the  availability  of  extensive  binomial 

^  .,     ^(CMIi) 

tables,  T      will  often  be  standardized  as  in  (3.5)  and  compared  to 

standard  normal  tables  since  the  normal  approximation  to  the  binomial 

when  p  =  1/2  is  satisfactory  even  for  fairly  small  sample  sizes. 

(F) 
The  asymptotic  null  distribution  of  T    in  the  discrete  case 

is  easily  shown  to  be  chi-square  with  2k  degrees  of  freedom.   This  is 

(F) 
also  the  exact  distribution  of  T    in  the  continuous  case.   It  follows 

that  the  approximate  slope  in  the  discrete  case  is  the  same  as  the 

exact  slope  in  the  continuous  case.   In  summary, 

4R(^)=4^hQ)=c^^(Q)-C^(Q)=2      L   X.{p.  Zn   2p.+(l-p.)£n  2(l-p.)}. 

In  order  to  derive  the  approximate  slopes  for  x   "^^  and  T^^\ 

consider  the  linear  combination  Zn"  X^^^  of  whicliT*'^^'"^  and  T*-^\Tre 

in 
, .  ■.  1 

special  cases.   The  variate  X^""^  has  an  asymptotic  normal  distribution 

i 

with  mean  -  n.  and  variance  -  n^  under  the  null  hypothesis.   It  follows 
directly  that 


V   ~  ^^Vi  "  2  ^"i   )/2^^"i 
is  asymptotically  standard  normal. 

Proposition  6.   The  approximate  slope  of  T*-^^  i 


s 
n 


11  1 
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Proof.       First,    to   get    b(9). 


.    .  1     .„  a„(i)         1„   a+1 

^(a)  —    (Zn    X  --Zn.       ) 

/n  1 


n 


2v        1 
vn  in 


2a+i 


1      ,      n. 

2/  2a 

/  n 


n  n  2  n 


1  /A.2a+l 

2  /^^n^ 


-)■     as   n  -»•  «>   . 

1  /    ^,2cx+l 

2  /     '\ 

(a) 
with  probability  one  [9].   Now,  since  T    is  asymptoticallv  standard 

n 

normal, 

c^^^    =    [b(6)]2 

_    r^,«+l  1   y"+li2.1   „,2a+l 

-    [ZX.      P.    -   2   ^.      ]    /j  EX. 

=    [ZX"-^\2P.    -   1)]2/ZX2"-^1    . 
11  1 

Letting  a    =   0   yields    the   approximate   slope   ofT 

^Qm(^>  =  t^\(2p.  -i)]2/k 

-1  Cy  ■) 

Letting  a  =  -—  yields  the  approximate  slope  of  T    , 

cI^\Q)    =    [ExJ(2p.  -  l)]2/k. 
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By  inspection  of  tlie  above  approximate  slopes,  it  is  apparent 

that  T      is  more  efficient  when  the  p.  are  proportional  to  the  X 

1  i 


(the  relative  sample  sizes)  and  T  '^'   is  more  efficient  when  th 


e  p .  are 


e  approximate 


inversely  related  to  the  A..   The  boundary  of  the  parameter  space  where 

*^X  ^^^  "  ^Om  ^^'^  ^^  """^  '^l  "  ^2  "  '■■  "  Pr'  ^^°"^^e^-  The  statistic 
T  is  more  efficient  than  T  ^  in  more  than  half  of  the  parameter 
space.  As  a  further  comparison,  e(^)(j(CMH)^  T  '*^^),  th 
efficiency  of  T      with  respect  to  T^^  ,  can  be  integrated  over  the 

parameter  space.   The  result  is  greater  than  one  which  again  supports 

(CMH) 
use  of  T      .It  should  be  noted,  however,  that  when  the  p.  are  pro- 
portional to  the  A^  both  tests  have  high  efficiencies  relative  to  when 
p^  are  inversely  related  to  the  A..   Therefore  T^^^  is  more  efficient  in 
a  region  of  the  parameter  space  where  both  tests  have  relatively  low 
efficiency.   This  is  a  good  property  for  T  '^  . 

An  "approximate"  likelihood  ratio  test  is  introduced  in 
Section  3.2.   A  statistic  which  is  equivalent  to  the  form  given  in 
Section  3.2  is 

1-1     1  i 

Proposition  7.   The  approximate  slope  of  x  ^^^^  is 

n 

4lr(^)  =  ^A.(2p.-1)2. 
Proof.   To  find  b(0)  of  Part  1  of  Theorem  1, 

-*  {4Z  ^^(P^  '2^    ]'^  as   n  ->  CO 


''■.9 


witli    probability    one    fO],      Tims, 

b(0)    =    {4ZA.(p.  -|)2}^  =   {EA.(2p.  -1)2}^- 

To    find    f(t)    of   Part    2   of    Tlieorem   1,    the   asymptotic   null    distribution 
of    T  is    required.      According    to   Oosterhoff    [33], 

P{Ez^   l{z.  >0}    >   s}   =   2"*^     Z      ('^)P{x^Ss} 

11  ■    ,      J  J 

J=l 

where  z.  is  a  standard  normal  random  variable.   Since,  under  the  null 
1 

hypothesis 

(X^'^  -4  n.)/^:/4  ^  z. 
n.    2   1     1      1 

1 

in  distribution,  it  follows  that 

P{t(^^'^^  .s}  =  ?{iT^^'^^h'>-s'}^2-hd)?{x'>-s'}  (3.11) 

n  n  J    J 

as  n ->  00  for  all  s.   It  follows  that  the  associated  density  function  is 
a  linear  combination  of  chi-square  densities.   The  result  of  Killeen  et  al, 
can  be  applied  to  verify  that 

£n[l  -  F(t)]  ~  -  I  t^     as   t  ->-  " 

where  F  is  the  asymptotic  null  distribution  of  T     .  Hence, 

n 

c^Lr(6>  =  [bO)]^  =  EA.(2p.  -1)^ 

Before  proceeding  to  a  further  comparison  of  approximate  slopes,  the 
slopes  are  summarized  in  the  following  listing. 


50 


Test  Approximate  Slope 

Fisher's  (T^^^)  2EA  {p.  £n  2p.  +  (1-p.)  £n  2(l-p.)} 

ill       1  1 

Likelihood  Ratio  (T^    )  2ZA.{p.  £n  2p .  +  (l-p.)£n  2(l-p.)} 

"Approximate  Likelihood  Ratio"  (T^^^^^)   EX  .  (2p  .  -  1)^ 

Sum  of  Chi's  (T^'^^)  hl^y^   (2p   -  1)]^ 

k    1     i 

Cochran-Mantel-Haenzel  (T^^^^)  i[ZA.(2p.  -  1)]^ 


Letting  A     =    A7   (2p.    -1),    it    is   easy    to    see    that    c'^^^e)    > 
111  AT.R       ^    - 


k 


c  ^  (6)  since   Z  A.  ^  rt^A.]  .   It  is  also  true  that  c^^^  >  c^^\^)  . 
X  ^^^  1    k    1  ALR  -   CMH^  ^ 

Let  B^  =  (2p   -1).   It  can  easily  be  shown  that 

XA.B^  -  ^[ZA.B.]^  =  f  E  Z  A  A . (B   -  B  )^ 
11    k   11     k  .^_.   1  J   1    j^ 

given  that  EA.  =  k.   Therefore  T^"^^^^  dominates  both  T*^^^^^  and  T^^^ 
with  respect  to  approximate  slopes. 

Approximate  efficiencies  of  t^^^^  and  t''^^'^^  with  respect  to 

(F)  (L{^) 

T     (and  equivalently  T    )  for  A^  =  A^  =  1  are  given  for  several  points 

in  the  parameter  space  in  Table  1.   In  this  case  of  equal  sample  sizes, 

T  ^   is  equivalent  to  t'''^^^   Table  2  gives  efficiencies  of  t*"*^^^^'  t^^-*' 

and  T   '    with  respect  to  T^^'"*  (and  equivalently  t^^*^^)  for  A   =  1/3, 

A2  =  5/3.   The  values  of  A^  and  A^  imply  that  the  second  test  is  based 

on  five  times  as  many  observations  as  the  first  test.   When  the  exact 

null  distribution  of  T^  '  ^  (binomial  with  parameters   E  n .  and  1/2)  is 

i=l  ^ 
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to  be  used  to  determine  significance  levels  it  is  more  appropriate  to 
employ  the  exact  slope,  c    (9),  rather  than  the  approximate  slope, 
c^j^jjj(U).   The  exact  efficiencies  of  T      relative  to  T     (and 
equivalently  T    )  are  given  in  Tables  1  and  2  in  parentheses. 

The  efficiencies  listed  in  Tables  1  and  2  support  several 
previously  made  observations: 

1.  The  statistic  T      dominates  T      and  T    with  respect 

to  approximate  slopes  (efficiencies). 

(CMH)  (x) 

2.  The  test  based  on  T      dominates  the  test  based  on  T 

when  the  success  parameters  are  proportional  to  the  sample  sizes.   The 

(X) 
test  based  on  T    is  more  efficient  in  the  reverse  case.   The  test 

(X) 
based  on  T    is  more  efficient  in  a  region  of  relatively  low  effi- 
ciencies for  both  T^^^  and  t^^''^^  . 

3.  Exact  and  approximate  slopes  are  not,  in  general,  equivalent. 
They  are  in  close  agreement  for  parameters  close  to  the  null  hypothesis. 

4.  All  of  the  tabled  efficiencies  are  at  most  one.   This  is 

(F)      n  R) 
expected  from  the  optimality  properties  of  T    and  r'^  given  by  Theorem  2 

and  Proposition  3.   A  value  of  one  is  achieved  only  for  the  exact  effi- 

ciency  ot  r      when  p   =  p  .   This  is  consistent  with  the  fact  that 

(CMH) 
T      IS  the  most  powerful  test  (and  the  likelihood  ratio  test)  when 

Pi  =  P2- 
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Table  1 
Efficiencies  of  t:^^^^\    t^^\  and  T*-"^^^-*  Relative  to 
T^^'^'*,  or  Equivalently,  to  T^'^^  A   =  X   =  1 


P2      -5       .6       .7       .8       .9      1.0 


Pi 


.8 


.9 


1.0 


- 

(0.497) 

(0.489) 

(0.474) 

(0.447) 

(0.377) 

- 

0.497 

0.486 

0.467 

0.435 

0.361 

— 

0.993 

0.972 

0.934 

0.869 

0.721 

(0.497) 

(1.000) 

(0.892) 

(0.773) 

(0.674) 

(0.540) 

0.497 

0.993 

0.879 

0.752 

0.644 

0.505 

0.993 

0.993 

0.976 

0.939 

0.876 

0.729 

(0.489) 

(0.892) 

(1.000) 

(0.951) 

(0.856) 

(0.697) 

0.4S6 

0.879 

0.972 

0.909 

0.799 

0.632 

0.972 

0.976 

0.972 

0.945 

0.888 

0.748 

(0.474) 

(0.773) 

(0.951) 

(1.000) 

(0.964) 

(0.831) 

0.467 

0.752 

0.909 

0.934 

0.874 

0.722 

0.934 

0.939 

0.945 

0.934 

0.892 

0.768 

(0.447) 

(0.674) 

(0.856) 

(0.964) 

(1.000) 

(0.932) 

0.435 

0.644 

0.799 

0.871 

0.869 

0.763 

0.869 

0.876 

0.888 

0.892 

0.869 

0.773 

(0.377) 

(0.540) 

(0.697) 

(0.831) 

(0.932) 

(1.000) 

0.361 

0.505 

0.632 

0.722 

0.763 

0.721 

0.721 

0.729 

0.748 

0.768 

0.773 

0.721 
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Pi 


Table  2 

Efficiencies  of  t(^""\  T^^\    and  T^^^^^  Relative  to  T^^'^^ 

(F) 
or  Equivalently,  to  T^   ,  A   =  1/3,  X   =  5/3 


Pj      -5       .6       .7       .8       .9      1.0 


.6 


.7 


1.0 


- 

0.828 

0.810 

0.778 

0.725 

0.601 

- 

(0.832) 

(0.826) 

(0.814) 

(0.791) 

(0.703) 

- 

0.497 

0.486 

0.467 

0.435 

0.361 

- 

0.993 

0.972 

0.934 

0.869 

0.721 

0.166 

0.993 

0.935 

0.867 

0.790 

0.646 

(0.165) 

(1.000) 

(0.957) 

(0.914) 

(0.872) 

(0.771) 

0.497 

0.867 

0.694 

0.604 

0.531 

0.426 

0.993 

0.993 

0.973 

0.935 

0.871 

0.723 

0.162 

0.893 

0.972 

0.921 

0.839 

0.685 

(0.162) 

(0.901) 

(1.000) 

(0.978) 

(0.938) 

(0.836) 

0.486 

0.981 

0.848 

0.925 

0.623 

0.490 

0.972 

0.984 

0.972 

0.937 

0.874 

0.727 

0.156 

0.737 

0.932 

0.934 

0.867 

0.714 

(0.156) 

(0.736) 

(0.964) 

(1.000) 

(0.982) 

(0.897) 

0.467 

0.934 

0.924 

0.815 

0.702 

0.550 

0.934 

0.954 

0.960 

0.934 

0.896 

0.732 

0.145 

0.576 

0.838 

0.904 

0.869 

0.731 

(0.145) 

(0.585) 

(0.872) 

(0.976) 

(1.000) 

(0.952) 

0.435 

0.830 

0.921 

0.861 

0.759 

0.601 

0.869 

0.896 

0.924 

0.916 

0.869 

0.736 

0.120 

0.420 

0.673 

0.805 

0.823 

0.723 

(0.121) 

(0.428) 

(0.751) 

(0.878) 

(0.962) 

(1.000) 

0.361 

0.660 

0.812 

0.827 

0.767 

0.629 

0.721 

0.756 

0.815 

0.845 

0.829 

0.721 
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3.4   Powers  of  Combination  Methods 

In  the  previous  two  sections,  competing  methods  were  compared 
with  respect  to  asymptotic  efficiencies.   Asymptotic  efficiencies  com- 
pare sequences  of  test  statistics  in  some  sense  as  the  sample  sizes  tend 
to  infinity.   Such  comparisons  may  or  may  not  be  applicable  to  situa- 
tions when  small  sample  sizes  are  encountered.   Therefore,  the  methods 
of  combinations  are  compared  in  this  section  with  respect  to  exact  power. 

As  mentioned  previously,  exact  power  studies  are  often  intractible, 
For  the  test  statistics  considered  here,  power  functions  are  not  obtain- 
able in  any  simple  form  which  would  allow  direct  comparisons  between 
competing  methods.   However,  through  the  use  of  the  computer,  it  is 
possible  to  plot  contours  of  equal  power  in  the  parameter  space.   From 
such  plots,  the  relative  powers  of  the  competing  methods  can  be  surmised. 
The  first  step  in  obtaining  the  power  contours  is  the  generation 
of  the  null  distributions  for  each  of  the  five  statistics:   T     ,  T    , 
T     ,  T    ,  and  T     .   Size  ot  =  .05  acceptance  regions  for  each  of 
the  statistics  for  varying  sample  sizes  are  shown  in  Figures  3-8. 
Acceptance  regions  for  tests  with  equal  sample  sizes 

(n^  =  n^  =  10,  15,  20,  30)  appear  in  Figures  3-6.   The  statistics 

^(LR)      ^(ALR)  ^  ^. 

i     and  i      define  very  similar,  but  not  identical  tests  for 

"l  ^   "2  "^  ^^'  ^^'    ^^'  ■^^'      ^'^^y  define  exactly  the  same  a  =  .05  accep- 
tance regions  in  all  four  cases,  and  will  therefore  yield  identical 

power  contours.   Fisher's  statistic,  T^^^  defines  a  test  similar  to  T^^^^ 

,  T,(ALR)  ^  (Y) 

and  1      for  n^  =  n^  =  10,  15;  in  fact,  T^    '    defines  the  same  a  =  .05 

acceptance  region  for  those  sample  sizes.   The  major  difference  between 
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T    and  the  two  likelihood  ratio  statistics,  T  ^   and  T     ,  is  that 

(F) 
T    has  many  more  attainable  levels.   For  sample  sizes  n  = n  = 20,  30, 

T    defines  different  ct  =  .05  acceptance  regions  than  T     and  T 

The  statistics  T      and  T    are  equivalent  for  n   =  n  . 

Figures  7-8  portray  acceptance  regions  for  cases  of  unequal 
sample  sizes  (n  =  10,  n  =  20  and  n  =  10,  n  =  50) .   The  difference 
between  T    and  T      is  apparent  for  the  case  of  unequal  sample  sizes. 
The  statistics  T     and  T      define  different  o.  =    .05  acceptance 
regions.   In  both  figures,  it  is  seen  that  T    ,  T     ,  and  T      define 
similar  regions. 

In  Section  1.3  it  was  stated  that  Birnbaum  [7]  has  shown  that 
combination  procedures  must  produce  convex  acceptance  regions  in  the 
(X    ,  X    ,  ...,  X    )  hyperplane  in  order  to  be  admissible.   Each  of 
the  acceptance  regions  in  Figures  3-8  appear  to  satisfy  this  convex- 
ity condition. 

The  acceptance  regions  given  in  Figures  3-8  are  not  exact 
a  =  . 05  size  regions.   They  are  the  nominal  acceptance  regions  which  are 
the  closest  to  size  ot  =  .05.   In  order  to  make  a  fair  comparison  among 
the  powers  of  the  competing  methods,  all  of  the  acceptance  regions  must 
be  of  exactly  the  same  size.   This  can  be  accomplished  by  admitting 
certain  values  of  (X    ,  X   )  to  the  acceptance  region  with  probabil- 
ities between  zero  and  one.   A  more  precise  definition  of  this  procedure 
follows.   Suppose 

PlT^^'*  <  t „}  =  .05  -  a, 
n .      X. 

1 

P{T^^''  <  t  }  =  .05  +  b, 
n .      u 

1 
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and  T    does  not  take  on  any  values  between  t.  and  t  .   Then  all  1 

n  i  u  n . 

1  1 

such  that  T    =  t.  are  included  in  the  acceptance  region  with  probabil- 
n .      X- 

1 

ity  one  and  T    =  t   is  included  in  the  acceptance  region  with  prob- 
n.     u 

1 

ability  a/a+b. 

The  power  of  the  i    test  is  one  minus  the  probability  that  T 
^  n. 

1 

fails  in  the  acceptance  region.   More  precisely,  define  the  power  of 

.th 
the  1    test  to  be 

n.Cp^.p^)  =1-  [P(T^^^  ^  t3^|(pj^,P2)}  + (a/a  +  b)  p(T^^^  =  t^|(p^,P2)}] 

i  i 

(1)  ^'^ 

n      (1)      n  -X    n      (2)      n  -x 

=   l-[l  E        (  n^)Pi    (1-Pi)       (    )P9    (1-Po) 

1 

n       (1)      n  -x^^^  n      (2)      n  -x^^^ 

+  (a/a+b)  Z  Z        (     )P^    (1-p^)        (     )p2    (l-p^) 

,  (1)   (2)   „,(i)_  ^   X  X 

(X    ,x    ):l^_  -t^ 

1 
For  each  test  statistic,  power  is  calculated  for  2500  values  of 


(P-ijPq)  in  the  alternative  parameter  space.   This  was  accomplished  with 
a  FORTRAN  computer  program.   A  data  set  consisting  of  these  calculated 
powers  is  then  passed  into  SAS  [4,16].   The  plotting  capabilities  of 
SAS  are  then  exploited  to  portray  contours  of  equal  power  in  the 


(Pj^.P2)  plane. 
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Figures  9-14  are  .90  power  contours  corresponding  to  the 

acceptance  regions  of  Figures  3-8,  respectively.   The  Cochran-Mantel- 

CCMH ") 
Haenszel  procedure,  T     ,  is  most  powerful  in  the  center  of  the 

parameter  space;  that  is,  when  p   and  p   are  nearly  equal.   This  is 
expected  since  T      is  uniformly  most  powerful  when  p   =  p.  for  any 
choice  of  sample  sizes.   The  statistic  T      is  clearly  inferior 
to  T     ,  X  ^^^  ,  and  T    in  the  extremes  of  the  parameter  space,  that 
is,  when  p   and  p   are  quite  different.   Further,  the  deficiency  of 
^(CMH)  ^Qjjjpgj-g^j  f.Q  ^{^g  other  methods  when  p  and  p  are  different  is 
larger  than  the  deficiency  of  the  other  methods  when  p   =  p^-   From 
Figures  9  -  12  it  can  also  be  seen  that  the  central  wedge  of  the  param- 
eter space  where  T      is  more  powerful  shrinks  as  tlie  sample  sizes 

(F) 
increase.   Fisher's  statistic,  T   ,  and  the  likelihood  ratio  statis- 
tics, T     and  j^^^\    have  similar  power.   Fisher's  method  gives 
slightly  more  power  in  the  central  region  of  the  parameter  space  while 
T     and  x^^^^^  are  slightly  more  powerful  when  p   and  p^  are  very 
different. 

For  unequal  sample  sizes  (Figures  13,  14),  T    and  T     yield 
power  contours  too  similar  to  be  separated  on  the  drawings.   The  approx- 
imate likelihood  ratio  test,  T     ,  has  almost  the  same  power  as  T 
and  T    ,  having  slightly  more  power  when  the  experiment  based  on  more 

observations  lias  tlie  larger  p.,  and  slightly  less  power  in  the  reverse 

(v)  ,        ^(CMH)   , 

case.   The  sum  of  chi's  procedure,  T    is  not  equivalent  to  T      when 

.  ,  ^(t:MH)  ^  . 
n   7^  n  .   The  power  contours  are  very  different  with  T      being  more 

powerful  wlien  the  larger  experiment  matches  with  a  large  p  .   The 

(x) 
statistic  T    is  more  powerful  in  the  opposite  case. 
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Figures  15  -  20  are  .60  power  contours  concomitant  with  the  .90 
power  contours  in  Figures  9-14.   The  comparison  of  competing  methods 
may  be  more  appropriately  made  for  low  powers.   When  all  of  the  powers 
of  the  tests  are  high  it  is  probably  unimportant  which  test  is  used. 
The  patterns  observed  in  the  .90  power  contours  are  virtually  the  same 
in  the  .60  power  contours,  however.   No  additional  information  is  appar- 
ent except  that  the  patterns  are  consistent  over  a  wide  range  of  powers. 
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Figure  3.   Acceptance  Regions  for  n   -  n   -  10. 
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Figure  4.   Acceptance  Regions  for  n   =  n 
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Figure  5.   Acceptance  Regions  for  n,  -  n 
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Figure  6.   Acceptance  Regions  for  n   -  n   -  30. 
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Figure  7.   Acceptance  Regions  for  n   =  10,  n   =  20. 
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Acceptance  Regions  for  n  =  10,  n  =  50. 


(i5 


-p(F)  -^(LR)    (ALR) 


t(cmh) 


Figure  9.   .90  Power  Contours  for  n   =  n   =  10. 
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Figure  10.   .90  Power  Contours  for  n   =  n   =15, 
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Figure  11.   .90  Power  Contours  for  n   =  n  =  20. 
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Figure  12.   .90  Power  Contours  for  n   =  n   =30. 
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Figure  13.   .90  Power  Contours  for  n   =  10,  n   =  20. 
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Figure  14.   .90  Power  Contours  for  n   =  10,  n   =  50. 


^2 

I. On 


j(F)  -|-(LR)  y(ALR) 


-j-(CMH) 


71 


P, 


Figure  15.   .60  Power  Contours  for  n   =  n   =  10. 
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Figure  16.   .60  Power  Contours  for  n   =  n   =  15, 


2 
.0- 


.9- 


T 


(F) 


j(LR)  j(ALR) 
-j-(CMH) 


73 
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on 


Figure  17.   .60  Power  Contours  for  n   =  n   =  20. 
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Figure  18.   .60  Power  Contours  for  n   =  n   =  30. 
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Figure  19.   .60  Power  Contours  for  n   =  10,  n   =  20. 
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Figure  20.   .60  Power  Contours  for  n   =  10,  n   =  50. 


77 


3  .  6   A  Syntliesis  of  Comparisons 


When  detailed  prior  knowledge  of  the  unknown  parameters  is 

(F) 
unavailable  the  class  of  competing  methods  can  be  restricted  to  T    , 

„(LR)   _(ALR)        (X)    „,       ^.   ,  ^   -.1 

T     ,  T     ,  and  T    .   These  methods  are  compared  wxth  respect  to 

various  criteria  in  previous  sections.   In  this  section,  the  results 
of  these  comparisons  are  synthesized  to  make  recommendations  concern- 
ing the  optimum  choice  of  method  for  various  situations. 

For  the  comparisons  in  the  previous  sections,  the  null  hypothesis 
considered  is  H  :  p   =  p   =  ...  =  P,  =  1/2.   The  most  general  alternative 
hypothesis  considered  is  H  :  p.  >  1/2  (strict  inequality  for  at  least 
one  i) .   In  some  situations,  it  is  reasonable  to  assume  that  the  success 
probability  is  consistent  from  experiment  to  experiment.   In  such  cases 

the  alternative  hypothesis  of  interest  is  H  :  p   =  p   =  ...  =  p   >  1/2. 

B   k    z  k 

A  third  alternative  hypothesis  of  possible  interest  is  H  :  p.  >  1/2 

^   J 

(exactly  one  j).   This  alternative  is  appropriate  if  the  researcher 
believes  that  at  most  one  p.  will  be  greater  than  1/2.   The  hypotheses 
H   and  H   are  probably  the  more  frequently  encountered  alternatives  in 
practical  situations. 

The  following  recommendations  are  based  on  evidence  presented 
thus  far  in  this  dissertation: 

1.  The  minimum  significance  level,  T  ,  has  good  power  versus 
the  \\  alternative.  It  performs  poorly,  however,  versus  other  alterna- 
tives. 

2.  The  Cochran-Mantel-Haenszel  statistic,  T     ,  forms  the 
uniformly  most  powerful  test  against  H  .   Its  use  is  therefore  indicated 
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whenever  it  can  be  assumed  that  the  p.  are  not  very  different.   The 

.   ^(CMH)     ^       •,   .   •,       ,  -,       .     .    ^ 

statistic  T      performs  relatively  poorly  versus  alternatives  in  the 

extremes  of  the  parameter  space  (Type  B  alternatives). 

(p) 

3.  Fisher's  combination,  T    ,  is  not,  in  general,  the  most 

powerful  test  versus  a  particular  simple  alternative  hypothesis.   Its 
power,  however,  is  never  much  less  than  that  of  the  optimum  test. 
Fisher's  method  gives  good  coverage  to  the  entire  parameter  space  and 
its  use  is  therefore  indicated  whenever  specification  of  the  alternative 
hypothesis  cannot  be  made  more  precisely  than  H  . 

4.  There  seems  to  be  no  compelling  reasons  to  recommend  the 

(x) 

use  of  the  sum  of  chi's  procedure,  T    ,  unless  it  is  known,  a  priori, 
that  the  p.  are  inversely  related  to  the  sample  sizes  of  the  individual 

binomial  experiments. 

(LR) 

5.  The  likelihood  ratio  statistic,  T     and  the  approximate 

■1  •,■■•,.   J     .       •   .    ^(ALR)   ,  ^.  .  .,       ^(F) 

likelihood  ratio  statistic,  T     ,  define  tests  very  similar  to  T 

They  obtain  approximately  the  same  powers  throughout  the  parameter  space. 

Choosing  among  these  three  statistics  then  depends  upon  which  yields 

significance  levels  with  the  greatest  ease  and  accuracy.   This  problem 

is  addressed  in  Section  3.7. 
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3. 7   Approximation  of  the  Null  Distributions 

^.  _,(F)   „aR)  ^(ALR) 
or  T    ,  r    ,  T 


In  Section  1.6  the  problem  of  obtaining  significance  levels  for 

(F) 
Fisher's  statistic,  T    ,  when  the  data  are  discrete  is  discussed. 

2      '  2 
Lancaster's  transformations,  X  and  X   are  introduced.   It  is  estab- 

m      m 

2      '  2 
lished  in  Section  1.6  that  X  and  X   both  converge  to  chi-squares  with 

m       m 

2k  degrees  of  freedom.   Although  Lancaster's  approach  can  be  expected 

to  yield  good  approximate  levels  for  large  sample  sizes,  the  degree 

of  accuracy  has  not  been  established  for  small  or  moderate  sample  sizes. 

Some  indication  of  the  accuracy  of  significance  levels  obtained  from 

2      '  2 
X  and  X   is  given  by  observing  the  mean  and  variance  of  these  variates. 
m      m 

Table  3  (page  79)  lists  the  means  and  variances  for  n  =  1,2,..., 20  for 

2      '  2 
X   and  X   when  applied  to  one  experiment.   Since  the  altered  form  of 
m      m 

(¥) 
T    will  be  compared  to  a  chi-squarc  distribution  with  2k  degrees  of 

2      '2 
freedom,  it  is  desirous  that  the  mean  and  variance  of  X  and  X   are  as 

close  as  possible  to  the  mean  and  variance  of  the  chi-square  distribution 

with  2  degrees  of  freedom,  which  are  2  and  A,  respectively.   For  n>3, 

the  mean  and  variance  of  X^  are  closer  to  2  and  A,  respectively,  than 

m 

the  mean  and  variance  of  x'^.   This  suggests  that  X  should,  in  general, 

m  "' 

'2 
be  a  more  accurate  approximation  than  X^  . 

(LR)     ,  ., 
In  Section  3.2,  tlie  likelihood  ratio  statistic,  T     ,  and  tlie 

approximate  likelihood  ratio  statistic,  i:''^^^\    are  introduced.   The 

(LR) 
necessary  regularity  conditions  can  be  shown  to  be  satisfied  for  T     , 

so  that  the  statistic  can  be  deemed  asymptotically  a  chi-square  with 

k  degrees  of  freedom.   As  previously  stated,  the  null  distribution  of 


Table  3 


'2      2 
Mean  and  Variance  of  Lancaster's  X   and  X 

m       m 
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'2 
Median  chi-square  (X   ) 

ra 


Mean  chi-square  (X  ) 

m 


n 

Mean 

Variance 

Mean   Variance 

1 

1.9808 

1.9753 

2.000    1.9218 

2 

1.9531 

2.8587 

2.8036 

3 

1.9394 

3.2223 

3.2419 

4 

1.9362 

3.3754 

3.4760 

5 

1.9385 

3.4514 

3.6101 

6 

1.9430 

3.5014 

3.6923 

7 

1.9481 

3.5428 

3.7462 

8 

1.9530 

3.5804 

3.7836 

9 

1.9574 

3.6151 

3.8110 

10 

1.9612 

3.6465 

3.8320 

11 

1.9645 

3.6748 

3.8486 

12 

1.9674 

3.6998 

3.8621 

13 

1.9698 

3.7218 

3.8733 

14 

1.9719 

3.7412 

3.8828 

15 

1.9738 

3.7582 

3.8909 

16 

1.9754 

3.7733 

3.8980 

17 

1.9768 

3.7866 

3.9042 

18 

1.9781 

3.7986 

3.9097 

19 

1.9793 

3.8092 

3.9145 

20 

1.9803 

3.8188 

3.9189 
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T      can  be  approximated  with  a  distribution  derived  by  Oosterhoff. 
Significance  levels  are  then  determined  by  the  relationship 

P{T^'^^^^  >  c}  =  2"^  E  (*?)  P{x^  >  c)  . 

j=l  ^  J 

The  null  density  functions  of  T^^\  T^^^\    and  t^^^^^  are 
plotted  in  Figures  21  -  22  for  1<.  =  2,  n  =n  =6.   These  plots  give  an 
indication  of  the  difficulty  of  approximating  the  respective  null 
density  functions.   More  extreme  (larger)  values  of  the  statistics  do 
not  always  occur  with  smaller  probabilities.   This  fact  gives  the  jagged 
appearances  for  the  density  functions.   This  lack  of  smoothness  causer, 
difficulty  in  approximating  a  discrete  density  with  a  continuous  one. 

The  remainder  of  this  section  contains  numerical  comparisons 
of  the  above-mentioned  approximations.   The  goal  is  to  choose  the  approx- 
imation which  yields  significance  levels  closest  to  the  exact  levels 
of  the  respective  statistic. 

Tables  A  and  5  correspond  to  the  density  functions  pictured 

(F) 
in  Figures  21-22.   Table  4  lists  the  possible  events  as  ordered  by  T 

2      '  2 
Lancaster's  approximate  statistics,  X  and  X   are  calculated  for  each 
'^'^  mm 

2      '2 
event.   Although  it  is  not  generally  true,   X   and  X  ^  maintain  the  same 

(F) 
ordering  of  events  as  the  uncorrected  T    .   Significance  levels  obtained 

2       '  2 
by  comparing  X  and  X   to  a  chi-square  distribution  with  four  degrees 
-'     *^     °   m      m 

of  freedom  are  then  compared  to  exact  levels.   The  inaccuracies  of  these 

approximations  are  then  reflected  in  the  columns  labeled  percentage  error. 

u   ^(LR)    , 
Table  5  gives  an  evaluation  of  the  approximations  given  by  T     and 

^(ALR)^   These  statistics  define  equivalent  tests,  but  yield  different 
approximations  to  the  exact  densities. 
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The  percentage  errors  given  in  Tables  4  and  5  tend  to  favor 

■       .■  .      t(LR)    ,  T,(ALR) 

Lancaster  s  approximations  over  the  approximations  to  T     and  i 

?  '  2 

Both  X  and  X   yield  generally  conservative  results  in  this  partic- 
m      m 

2 
ular  case.   The  mean  chi-square,  X  ,  is  somewhat  more  accurate  than 

m 

'2 
the  median  chi-square  X 

m 

All  of  the  approximations  can  be  expected  to  improve  as  the 
sample  sizes  increase.   To  indicate  the  behavior  of  the  contending 
approximations  for  increasing  sample  sizes,  nominal  c  =  .05  and  ot  =  .01 
values  for  each  statistic  are  given  in  Tables  6-8  for  n^  =  n^  =  3,4 , 5,  . . 

The  data  in  Tables  4-8  indicate  that  Lancaster's  approxima- 

^(LR)    ,  „(ALR)    „, 
tions  clearly  dominate  the  approximations  to  T     and  1      .   ine 

2         '2  (F) 

optimal  choice  then  becomes  either  the  X  or  the  X   correction  to  T 
■^  mm 

2      '2 
Table  6  gives  no  clear  indication  as  to  whether  X  or  X 

yields  a  better  approximation.   Both  statistics  give  large  errors  for 

small  sample  sizes.   Both  statistics  yield  errors  less  than  12%  for 

both  ci  =  .05  and  ct  =  .01  levels  for  n>16. 

2 
The  superiority  of  the  mean  chi-square,  X",  over  the  median 


chi-square,  X   ,  becomes  clear  for  k=3.   Table  9  gives  the  nominal 
m 

9       '  2 
a  =  ,05  and  a  =  .01  values  for  X"  and  X   for  k  =  3,   n   =  n   -  n   - 

mm  i  z  J 

2 
2, 3, 4..., 10.      The  mean   chi-square,    X    ,    is  more   accurate   in   all    cases 

m 

but    two    (a   =    .01,    n  =  8   and  ct   =    .05,    n  =  5)  . 
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Figure  22.   Density  Functions  of  T*-^^^  and  t*'^^^''  for  n   =  n   =6. 
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EvenC 

6,6 
6,5 

6,4 

6,3 

5,5 

6,2 

6,1 

6,0 

5,4 

5,3 

5,2 

4.4 

5,1 

5,0 

4,3 

4,2 

4,1 

4,0 


Table  4 


Lancaster's  Approximations  to  T 


Approximate 
Level 

.000374 

.003209 
.002894 

.009954 
.009541 

.01969 
.01941 

.02557 
.02099 

.02852 
.02843 

.03354 
.03353 

.03517 
.03517 

.07445 
.06511 

.1396 
.1258 

.1948 
.1779 

.2038 
.1894 

.2248 
.2062 

.2344 
.2152 

.3600 
.3441 

.4804 
.4646 

.5415 
.5254 

.5603 
.5441 


(F) 


for  k  =  2,  n  =  n 


X^ 
m 

'2 
,  X 

ra 

20 

.6355 

15 

.8629 

16 

.0951 

13 

.2872 

13 

.3847 

11 

.7041 

11 

.7376 

11 

.0904 

11 

.5546 

10 

.8316 

10 

.8393 

10 

.4468 

10 

.4477 

10 

.3335 

10 

.3335 

8. 

.5146 

8, 

.8442 

6, 

,9315 

7. 

,1972 

6. 

.0590 

6. 

,2988 

5. 

9389 

6. 

1338 

5. 

6743 

5. 

9072 

5. 

5609 

5. 

7930 

4. 

3558 

4. 

4867 

3. 

4833 

3. 

5884 

3. 

0985 

3. 

1968 

2. 

9852 

3. 

0826 

Exact 
Level 

.0002441 
.003174 

.01050 

.02026 

.02905 

.03638 

.03931 

.03979 

.08374 

.1423 

.1863 
.2412 
.2588 
.2617 
.4082 
.5181 
.5620 
.5693 


Percent 

Error 

53 

.2 

1 

.1 

-8 

.8 

-5 

.2 

-9 

.1 

-2 

.8 

-4 

.2 

-21 

.0 

-27 

.7 

-21 

.6 

-21 

.9 

-14 

.7 

-14 

.7 

-11 

.6 

-11 

.6 

-11 

.1 

-22, 

.2 

-1. 

.9 

-11, 

.6 

4. 

,6 

-4. 

,5 

-15. 

5 

-21. 

5 

-13. 

1 

-20. 

3 

-10. 

4 

-17. 

7 

-11. 

8 

-15. 

7 

-7. 

3 

-10. 

3 

-3. 

6 

-6. 

5 

-1. 

6 

-4. 

4 

Table   4    (Continued) 
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Event 

2    '2 
X  ,  X  ^ 

m   m 

Approximate 
Level 

Exact 
Level 

Percent 
Error 

3.3 

2.7726 
2.8397 

.5966 
.5850 

.6670 

-10.6 
-12.3 

3,2 

1.9001 
1.9414 

.7541 
.7465 

.8135 

-7.3 
-8.2 

3,1 

1.5154 
1.5498 

.8239 
.8178 

.8721 

-5.5 
-6.2 

3,0 

1.4020 
1.4356 

.8438 
.8380 

.8818 

-4.3 
-5.0 

2,2 

1.0276 
1.0431 

.9056 

.9032 

.9368 

-3.3 
-3.6 

2,1 

.6429 
.6514 

.9582 
.9572 

.9807 

-2.3 
-2.4 

2,0 

.5295 
.5372 

.9706 
.9698 

.9880 

-1.8 
-1.8 

1,1 

.2582 
.2598 

.9924 
.9923 

.9968 

-0.4 
-0.5 

1,0 

.1448 
.1456 

.9975 
.9975 

.9998 

-0.2 
-0.2 

0,0 

.03142 
.03142 

.9999 
.9999 

1.0000 

0.0 
0.0 

Table    5 


^(LR)  ,    ^(ALR) 

Approxmacioiis    to   T  and    T 

for   k  =  2,    n     =  n     =  6 
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Event 

^(LR)   ^(ALR) 

Approximate 

Exact 

Percent 

Level 

Level 

Error 

(6,6) 

16.44 
12.00 

.002 
.000886 

.0002441 

0.0 
-63.7 

(6,5) 

11.23 
8.667 

.0036 
.004901 

.003209 

12.5 
52.7 

(6,4) 

8.977 
6.667 

.0111 
.01383 

.0150 

5.7 
31.7 

(6,3). (6,2) 

8.318 

.0156 

.0310 

-49.7 

(6,1), (6,0) 

6.000 

.0196 

-36.8 

(5,5) 

5.822 
5.333 

.0544 
.02783 

.0398 

36.7 
-30.1 

(5,4) 

3.591 
3.333 

.1660 
.08117 

.0837 

98.3 
-3.0 

(5,3), (5,2) 

2.911 

.2333 

.2068 

12.8 

(5,1), (5,0) 

2.667 

.1171 

-43.4 

(4,4) 

1.359 
1.333 

.5069 
.2525 

.2617 

93.7 
-3.5 

(4,3), (4,2) 

.6790 

.7119 

.5693 

25.0 

(4,1), (4,0) 

.6667 

.3862 

-35.2 

Remainder 

0 
0 

1.0000 
1.0000 

1.0000 

0.0 
0.0 
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(F)    (LR)      ,  ,,(ALR)  ,  ..  .  .1 

Tlie  statistics  T    ,  T     ,  and  1      define  very  sxmiiar 

tests.   In  Section  3.6  it  is  concluded,  therefore,  that  the  choice 

among  these  three  statistics  should  depend  upon  which  affords  the  best 

approximation  to  its  null  distribution.   The  evidence  of  this  section 

2  .        (F) 

indicates  that  Lancaster's  mean  chi-square  (X  )  approximation  to  T 

m 

is  the  best  choice.   Even  this  approximation  yields  large  errors  for 
small  sample  sizes.   Tables  10  and  11  give  nominal  «  =  .05  and  «  =  .01 
levels  for  IlL   ,  an  equivalent  and  more  convenient  form  of  T    for 
k  =  2  and  k  = 3.   The  exact  significance  levels  are  also  given. 
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Nominal  ot  =  .05 


a 

n 

HL^^) 

3 

.0156 

4 

.0195 

5 

.0254 

6 

.0150 

7 

.0142 

8 

.0209 

9 

.0195 

10 

.0107 

11 

.0164 

12 

.0155 

13 

.0178 

14 

.0173 

15 

.0176 

16 

.0154 

17 

.0168 

18 

.0154 

19 

.0150 

20 

.0155 

21 

.0133 

22 

.0153 

23 

.0155 

24 

.0134 

25 

.0142 

26 

.0145 

27 

.0135 

28 

.0148 

29 

.0154 

30 

.0144 

05 


Tabl 

e  10 

a  =  .01 

Events  for  n  L 

'\^-2 

Exact 
Level 

a  = 

.01 

Exact 
Level 

.0156 

.0352 

.0039 

.0039 

.0498 

.0010 

.0010 

.0398 

.0017 

.0032 

.0365 

.0039 

.0078 

.0412 

.0039 

.0088 

.0436 

.0020 

.0067 

.0336 

.0030 

.0067 

.0440 

.0029 

.0070 

.0478 

.0032 

.0072 

.0482 

.0033 

.0100 

.0427 

.0026 

.0094 

.0442 

.0027 

.0093 

.0446 

.0024 

.0087 

.0482 

.0032 

.0095 

.0493 

.0029 

.0096 

.0452 

.0027 

.0098 

.0492 

.0027 

.0098 

.0453 

.0025 

.0095 

.0448 

.0022 

.0093 

.0452 

.0027 

.0097 

.0477 

.0024 

.0098 

.0479 

.0025 

.0091 

.0500 

.0027 

.0100 

.0474 

.0028 

.0099 

.0469 

.0027 

.0100 

.0480 

.0025 

.0098 

.0479 

.0023 

.0090 
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Table  11 


Nominal  a  =  .05  and  a  =  ,  01  Events  for  HL    ,  k  =  3 


a  =  .05 


a  =  .01 


n 

nL<'> 

Exact 
Level 

nL<^> 

Exact 
Level 

3 

.01563 

.0430 

.001953 

.001953 

4 

.006104 

.0229 

.002686 

.0076 

5 

.007813 

.0419 

.001099 

.0052 

6 

.007851 

.0458 

.001709 

.0098 

7 

.006866 

.0357 

.001369 

.0091 

8 

.007589 

.0468 

.001193 

.0089 

9 

.006022 

.0471 

.001087 

.0093 

10 

.006327 

.0499 

.001127 

.0095 

CHAPTER  IV 
APPLICATIONS  AND  FUTURE  RESEARCH 

4. 1   Introduction 

In  the  first  three  chapters,  the  combination  problem  was 
approached  from  the  standpoint  of  hypothesis  testing.   It  is  often 
desirable  to  consider  problems  in  statistical  inference  from  the 
standpoint  of  estimation.   In  Section  4.2,  such  an  approach  is  con- 
sidered.  Confidence  regions  based  upon  previously  considered  testing 
methods  are  derived. 

The  remaining  sections  of  Chapter  IV  discuss  potential  research 
problems  for  which  the  combination  methods  studied  in  this  dissertation 
may  be  applied  to  gain  solutions. 


4.2   Estimation:   Confidence  Regions  Based  on 
Non-parametric  Combination  Methods 


Consider  the  general  combining  problem  where  the  null  hypoth- 
eses of  concern  are  parametric.   That  is,  the  i   null  hypothesis  can 

be  written  H  :  9  =d  ,  where  9,  is  an  unknown  parameter  and  d .  is  a 
i    i    1         1  1 

specified  value.   Since  the  test  statistic  X  ^      typically  depends  on 

the  particular  value  of  d.,  write  X^^^  =  X^^^(d.).   The  observed  sig- 
*^  1  1 

nigicance  level  for  the  i    test  can  then  be  written  L    (d  )  = 

1  -  F     (X    (d  ))  where  F.  ,   is  the  cumulative  distribution  function 
i,d .       i  i>d . 
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of    X         (d    )    under    the    null    hypothesis    11.:    ^.  =(i..       Following    the    same 
1  111 

notation,    Fisher's   statistic   can   be  written 

}^ 

I'-^^d    ,d    ,...,d    )    =   -2   Z    Jin   L^-'^^d.) 

1=1 

and  Tippett's  statistic,  the  minimum  significance  level,  can  be  written 

T^'"^(d,,d_,...,d,  )  =  min  L*'^-'(d.). 
12k      .        1 

1 

A  100(1-0')%  joint  confidence  region  for  the  parameter  vector 

^  =  (^  .^-^j'-'.^  )  can  be  obtained  by  inverting  Fisher's  statistic 
12      k. 

T    (d^,...,d  ).   Following  the  usual  procedure  of  inverting  test 

X         K. 

statistics,  the  resulting  region  is  given  by 

Sj.  =  {(d^,d2,...,dj^):   -2  Z  In   L^^^d.)  <  x\\^(^-^)^ 

i=l 

2 
where  X   (1-*^)  is  the  l-ot  quantile  of  the  central  chi-square  distribu- 
tion with  2k  degrees  of  freedom.   Regions  can  likewise  be  obtained  by 

r„(ni)   .  ,  , . 
inverting  T    yielding 

S   =  {(d,,d„,...,d,):  L.(d.)  >  1  -  (1-^)^^^,  i  =  1,2, ...,k}. 
m      12      k    1   1 

Any  of  the  other  non-parametric  combining  methods  introduced  in  Sec- 
tion 1.2  can  likewise  be  inverted  to  yield  confidence  regions. 

If  the  6   are  one-dimensional  and  functionally  independent, 
i 

then  the  region  S   is  the  usual  rectangular  confidence  region,  and 

°  m 

therefore  is  easily  computed  and  displayed  in  application.   The  inequal- 
ity which  defines  S^  cannot  generally  be  simplified.   However,  as  will 
^  F 

be  shown,  the  region  S„  has  a  simple  explicit  form  for  location  param- 

F 

eters  of  negative  exponential  distributions.   While  the  region  S^  is 
not  as  simply  computed  and  displayed  as  S  ,  it  can  be  obtained  and 
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displayed  using  a  computer  program  package  capable  of  computing 
significance  levels  (that  is,  capable  of  evaluating  probability  inte- 
gral transformations)  and  constructing  plots.   The  Statistical  Analy- 
sis System  (SAS)  [4,15]  is  one  such  package  which  can  readily  evaluate 
the  probability  integral  transformation  for  common  distribution  func- 
tions (including  normal,  t,  F,  and  gamma)  with  integrated  data  manage- 
ment and  plotting  capabilities.   All  the  plots  in  the  examples  of  this 
section  were  obtained  by  hand  or  by  using  SAS.   The  amount  of  work 
involved  in  constructing  the  plots  is  essentially  the  same  as  that 
required  to  plot  response  surface  contours  for  second  order  models. 
The  relative  performance  of  confidence  regions  is  directly 
related  to  the  relative  performance  of  the  corresponding  tests  whose 
inversion  yields  the  confidence  regions.   Specifically,  a  most  powerful 
test  yields  confidence  sets  which  minimize  the  coverage  probability  of 
"non-true"  parameters  [21].  The  results  of  the  comparisons  among  combin- 
ing procedures  given  in  this  dissertation  can  therefore  be  translated 

into  comparisons  of  procedures  for  obtaining  joint  confidence  regions. 

^(F) 
The  results  of  Chapters  1  and  II  tend  generally  to  support  T    as  a 

procedure  which,  though  not  most  powerful,  has  good  power  (relative  to 

a  most  powerful  test)  over  most  of  the  combined  alternative  space. 

We  thus  conclude  that,  if  8  =  (^  j^.  •  •  •  .^j^)  is  the  "true"  parameter  value, 

then  S   should  have  relatively  low  coverage  probability  of  0'  for  all 

6 '  j:  B  .   It  will  be  seen  that  S   is  often  similar  to  the  region  S^^ 

based  on  the  likelihood  ratio  statistic. 

The  test  statistic  T    generally  has  quite  good  relative  power 

against  points  in  the  combined  alternative  space  for  which  only  one  (or 
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at  least  a  small  number)  of  H  , ...,H   are  false  (Type  C  alternative), 

but  does  not  have  good  relative  power  against  points  for  which  all,  or 

nearly  all,  of  H,,...,H,  are  false.   Therefore,  S   should  have  quite 
Ik  m  ^ 

low  relative  coverage  probability  of  6'  =  (0'  ..,,6')  if,  say,  6'  ?=  9 

and  Q\  ^  Q . ,    1  =  2,...,k,  but  S   may  have  a  rather  high  relative  cover- 
i    1  ra 

age  probability  of  6'  if  6'  *   Q.,    i  =  l,...,k. 

Several  examples  of  confidence  regions  based  upon  non-parametric 
combining  methods  are  now  presented. 


Example  1.   Shift  Parameters  for  Negative  Exponentials 

Let  x.,,...,x.    be  a  random  sample  from  a  population  with 
il      in.  ^  '^    '^ 

-(X..-0.) 

density  function  f(x..,6.)  =  e   ^-^      "^  ;  x.  .  >  Q  .  ;  i  =  1,  .  .  .  ,k. 

ij   1  '   ij    1 

We  wish  to  obtain  100(l-oi)%  confidence  regions  for  9  =  (d  .  ,  .  .  .  ,6  )  . 

The  likelihood  ratio  test  of  H.:  0 _  =  d.  versus  the  alternative 

11     1 

A.:  y.  >  d.  is  to  reject  H.  if  X.,,,  -  d.  is  larger  than  a  prescribed 
iiii  ii(l)i       ^  ^ 

value,  where  X.  ,  ,  is  the  smallest  order  statistic  from  the  i   sample. 

(i)  ( - )  -n.  (X    -d . ) 

Taking  X^  '^  (d^)  =  X^    -  d^,  one  obtains  L^^'^(d.)  =  e   ^   ^^^   ^ 

and  thus 

Sp=((d^,...,d^^):  2Xn.(X.^^^-d.)<x^,(l-).  d.<X.^^^,  i  =  l,...,k}. 

It  is  of  incidental  note  that  Fisher's  combined  test  for  this  problem 
is  the  same  as  the  likelihood  ratio  test  based  on  the  joint  likeli- 
liood  of  the  k  samples. 
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The  (one-sided)  rectangular  region  S   has  tlic  form 

S^=  ((d.,...,d^):  X,(,)-c.<d.<X.^^^,   i  =  l....,k} 

-i  1/k 

where  c   =  -n    Xnfl  -  (1-ct)    J.   Figure  23  shows  S^  and  S   depicted 

for  k  =  2,  n  =  n  = 10,  x 


^^^^  =2.1393,  and  x^^^^  =1.0725. 


1    2 

Much  attention  has  been  paid  in  the  literature  to  the  construc- 
tion of  confidence  intervals  with  the  goal  of  minimizing  the  width  of 
the  intervals.   The  direct  k-dimensional  analog  of  this  notion  is  the 
construction  of  confidence  regions  with  minimized  area  in  the  parameter 
space.   The  relative  areas  of  S   and  S   for  k  =  2  and  n  =  n^  are  pre- 
sented in  Table  12  (below).   It  is  seen  that  S„  has  smaller  area  than 

r 

S   for  several  values  of  o. . 
m 

Table  12 

Ratio  r =  (Area  S  )/(Area  S^)  for  Location  Parameters 
m         F 

of  Double  Exponential  Distributions,  when  k  =  2 


.10 

.05 

.025 

.01 

.005 

1.17 

1.20 

1.23 

1.27 

1.20 

Example  2.   Means  for  Normal  Distributions 

For  this  example  9  =  (Pt,...,^, )  and  the  x   , . . . ,x    , 

Ik  li      i^i 

i  =  l,...,k  are  assumed  to  be  random  samples  from  normal  distributions 

2 
with  mean  u      and  known  variances  o,,  i  =  l,...,k.   The  standard  joint 

confidence  region  for  9  is  the  ellipsoid 


SlR=^(^1'---'V=  ^n.(X.-d.)2/o;<x^(l-a)} 
which  is  obtained  by  inverting  the  likelihood  ratio  statistic. 


(4.1) 


lOi 


e. 


.0725 


0.7049 


S 


m 


S     -  S 


Figure    23. 


Confidence  Regions  S   and  S„  =  S^ „  for  the  Shift  Parameters 
of  Two  Negative  Exponential  Distributions. 
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The  region  based  on  Fisher's  combin^ilion  is 

Sp  =  {(d^,....dj^):  -2Z  In   2  [1  -  $(  |  X.  -  d  .  | /o. )  ]  <X2j^(l-^)} 

and  the  rectangular  region  based  on  T    is 

S   =  {(d,,...,d,  ):  n.(X.  -d.)^/oJ<xJ((l-a)^/'',  i  =  l,...,k}. 
m      1      k    111    il 

There  is  no  practical  advantage  to  using  S   instead  of  S^^^^  for 

this  example  because  S   is  at  least  as  easily  obtained  and  presented 

LR 

as  S  .   Further,  the  confidence  coefficients  for  both  S   and  S    are 
known  exactly  for  this  example.   Our  purpose  in  presenting  S^  is  to 
show  how  it  compares  with  a  theoretically  optimal  procedure.   The 
graphs  in  Figure  24  show  that  S   and  S    are  similar,  which  concurs 
with  comparisons  given  in  Chapter  1  from  a  testing  viewpoint. 

The  fact  that  the  boundary  of  S„  is  exterior  to  S   along  the 

F  m 

middles  of  the  sides  of  S   and  interior  to  S   in  the  corners  supports 

m  m 

previous  statements  regarding  relative  coverage  probability  of  the  two 
regions.   That  is,  S   will  tend  to  have  a  lower  probability  than  S   of 
containing  "non-true"  points  (9',^')  which  are  diagonally  away  from  the 
"true"  (e,,9„),  and  conversely  for  points  (9|,9')  which  are  horizontally 

or  vertically  away  from  (9  ,9  ) . 

2 
If  the  o   are  unknown,  the  exact  100(1-^^)%  regions  for 

1 

(F)    ,  ^(m) 
0  =  (M,,...,Vi,  )  can  still  be  obtained  by  inverting  T     and  T 
1      k 

Exact  100(l-a)%  confidence  regions  for  small  samples  based  on  the  like- 
lihood ratio  (analogous  to  4.1)  are  not  easily  obtained  because  the 
probability  distribution  involved  is  a  sum  of  F  statistics.   It  there- 
fore may  be  preferable  to  use  S^  or  S   in  this  case.   The  relative 
■'  >^  F     m 


9: 
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LR 


•0, 


Figure  24.   Confidence  Regions  S  ,  S  ,  and  S    for  Means  of  Two 


Normal  Distributions. 
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2 
appearance  of  S   and  S   when  the  o.  are  unknown  is  essentially  the 

same  as  shown  in  Figure  24. 

Now  suppose  the  M.  have  a  common,  but  unknown,  value  y,  and 

suppose  the  o  are  unknown.   An  exact  100(l-a)%  confidence  set  for  y 

(F)      (m)    .  . 
can  once  again  be  obtained  by  inverting  T    or  T    ,  giving 


S   =  {d:  -2E  Zn   2[1  -  F.CX^'-^d))]  <X2j^(l-^)} 


,   =  Id:  -zz-  )tn  zli  -  r  .  C'        ^j^^i  -- - 

and 

S  =  {d:  2[l-F.(X^'-\d))]  >  (1-")^'^'',  i=  l,...,k}, 
m  1 

where  X    (d)  =  v^  I x  -dl/s   and  F.  is  the  distribution  function  of 

'  i      1      1 

students  t  on  n.  -1  degrees  of  freedom. 

1 

It  should  be  noted  that  S^  and  S   are  not  necessarily  inter- 

t       m 

vals.   The  set  S   may  be  a  union  of  intervals,  and  the  set  S^  is  the 

1/k 
intersection  of  the  k  individual  lOO(l-a)    %  intervals  that  could  be 

constructed  from  the  individual  samples.   However,  the  failure  of  Sp 

or  S   to  be  an  interval  is  an  indication  that  the  model  is  not  valid, 
m 

that  is,  that  not  all  the  means  really  are  equal.   Thus  the  failure  of 

S   or  S   to  yield  an  interval  is  more  a  fault  of  the  assumed  model  than 
F     m 

of  the  statistical  procedure. 

It  should  be  noted  that,  generally,  if  one  desires  to  estimate 
a  parameter  6  common  to  each  of  k  populations,  and  if  nuisance  param- 
eters vary  from  one  population  to  the  next,  then  n  combination  of  k 
individual  tests  for  6  can  be  inverted  to  obtain  a  combined  confidence 
set  for  6. 
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Example  3.   Mean  and  Variance  for  a  Normal  Distribution 

Suppose  X  ,...,x   is  a  sample  from  a  normal  distribution  with 

2 
mean  y  and  variance  o   .      The  usual  test  of  H  :  y  =  m,  a=  t  versus  the 

alternative  A  :  M  ^^  m,  a  =  t,  where  ra  and  t  are  known  and  specified,  is 
to  reject  H   if  X    =  /n|x-ml/t  is  large.   Also,  the  usual  test  of 
H  :  o  =  t  versus  the  alternative  A  :  a  ?:  t  is  to  reject  H^  if 
X    =  Z(x.  -  x)  /t   is  either  too  small  or  too  large.   Under  the  com- 
bined null  hypotheses  H  =  H   n  H  ,  the  distributions  of  X    and  X 

2 
are  independent.   Thus  an  exact  100(1-^)%  confidence  region  for  (y , o  ) 

is  given  by 

S   =  {(m,t^):  -2  Jin  L^^\m,t)  -2  £n  L^^^t)  <x^l-^)}      (^-2) 
F  ^ 

where  the  significance  levels  L    and  L     (of  X    and  X   )  are 

2 
obtained  from  standard  normal  and  x^_^   distributions,  respectively. 

The  region  based  on  T    is  given  by 

S  ={(m.t^):  min(L^^^m,t),  L^^^*  (t) )  >  1  -  (1-a)  ^^^} 
m 

=  {(n..t'):    x(^\m.t)<z,    X^d -^(1-«)S  <  ^^'^t)       <        xjcfd-)^)} 
=  [(m,t^):    X-z    t/n^/2<m<x+z    t/n^/^    S^/x^(|(l-")S 

<  t^<s^/xj(l-^(l-")^} 
where  z  is  the  1  -  ^^ ^        quantile  of  the  standard  normal  distribution 


-.2 


an 


d  S   =  Z(x.  -x)  .    Note  that  the  region  S^  is  not  rectangular  due  to 


X 

2  2 

tlie  functional  dependencies  between  0^  =  (p  ,  o  )  and  0  ^  =  o  .   Graphs  of 

S   and  S   are  shown  in  Figure  25  along  with  a  graph  of  S^^^,  the  region 
F      ni 

based  on  the  likelihood  ratio  statistic.   The  confidence  coefficient 


cr 
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Figure  25.   Confidence  Regions  S  ,  S  ,  and  S    for  the  Mean  and 

m    r        LK 

Variance  of  a  Normal  Distribution. 
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for  S    is  noL  exact,  however,  because  it  was  obtained  from  the 
LR 

asymptotic  distribution  of  the  likelihood  ratio  statistic. 


Example  4.   Shift  and  Scale  Parameters  of  Negative  Exponential 
Distribution  with  Censored  Data 

Let  x.,s...x,  ,  denote  the  first  r  order  statistics  of  a 
(1)     (r) 

sample  of  size  n  from  a  population  with  density  function 
f(x;B,o)  =  o-lg-(^-B)/o^  x>0.   Then,  under  the  hypothesis 
H:  B  =  m,  o  =  t,  the  statistics 


X^^^  =  2n(x^^^  -m)/t  (A. 3) 


and 


r 

i=l 

2 
are  independent  x   on  2  and  2(r-l)  degrees  of  freedom,  respectively. 

Thus  an  exact  100(l-ot)%  confidence  region  for  (B,o)  is  given  by  (4.2) 
where  L^^\m,  t)  and  L^^\t)  are  the  levels  of  x    and  x    given  in 
(4.3)  and  (4.4). 

Optimality  of  Fisher's  combination  of  X    and  X    is  dis- 
cussed by  Perng  [38]. 


Example  5.   Mean  Differences  and  Variance  Ratios 

Let  x  ,...,x   and  y  ,...,y   be  independent  random  samples  from 
1 '  '  '  '  m       1       n 

two  normal  populations  with  respective  means  M^  and  \i ^   and  respective 
variances  oj  and  o^.   A  test  of  H:  ^^=^2'  a^  =  o^  versUs  the  alternative 

A:  M  ^  \^      or  o  a:  o"  is  obtained  by  combining  the  usual  t-test  for  test- 

2  _   2 
ing  H  :  M,  =V      and  the  usual  F-test  for  testing  H^:  o^  -  o^ .   This 
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procedure  was  proposed  by  Sukhatme  [41]  and  was  shown  by  Perng  and 

Littell  [38]  to  be  asymptotically  as  efficient  as  the  likelihood  ratio 

2   2 
test.   Exact  100(1-'^)%  confidence  regions  for  (^'2"^!'  °2^°1^  ^^^ 

obtained  by  inverting  the  combined  test,  yielding 

Sp  =  {(d^,d^):  -2  £n  L^^\d^)  -2  ^n  L^^^d^)  <  xj) 

where  L    (d  )  is  the  two-sided  level  of  the  t-test  for 

H  •  u   -  u   =  d   and  L    (d^)  is  the  two-sided  level  of  the  F-test 
12    1    1  2 

2   2 
for  H^:  02/°!  "  ^2' 

Example  6.   One-sided  Bounds  for  Multivariate  Means 

Consider  now  a  sample  of  size  n  from  a  multivariate  normal 
population  with  mean  vector  M  and  known  covariance  matrix  Z.   In  view 
of  the  computational  difficulties  involved  with  computing  the  likeli- 
hood ratio  statistic  for  a  one-sided  test  about  U,  Brown  [9]  proposed 
Fisher's  combination  of  k  univariate  one-sided  tests  as  a  one-sided 
test  about  y.   Brown's  approach  is  summarized  in  Section  1.5.   This 
procedure  can  be  inverted  to  obtain  approximate  lOO(l-ct)  one-sided 
confidence  regions  for  y  which  have  distinct  computational  advantage 
over  regions  based  on  likelihood  ratio  statistics  for  the  one-sided 
problem. 

Example  7.   Variance  Components 

Consider  the  random  one-way  classification  model 

Y   =  M  +  a  +  e.  .  (i  =  1,.  .  .,t;  j  =  1, . . .,n)  where  y  is  a  fixed 

2  .  1 
parameter,  a.  is  normal  with  zero  mean  and  variance  o  ,  e  xs  normal 
^1  a   ij 

2 
with  zero  mean  and  variance  o   and  all  a.  and  £..  are  mutually 

t-  1  X  J 
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independent.   The  analysis  of  variance  sums  of  squares  can  be  used  to 

2      2 
construct  a  joint  100(1-<J')%  confidence  region  for  o  and  o        given  by 

Sp  =  {(d^.d^:  -2  iln  L^^^^d^.d^  -2  ^n  L^^^d^)  <X(^)} 

where  L    (d  ,d  )  is  the  observed  two-sided  level  of  SSA/(d^+nd  )  and 
a   ^  £   a 

L^^^(d  )  is  the  observed  two-sided  level  of  SSE/d  .   Both  L^^^  and  L 

2 
are  obtained  from  X   tables.   Inversion  of  combined  analysis  of  var- 

2 
iance  x   statistics  could  be  useful  for  more  complex  orthogonal  designs. 


Example  8.   Binomial  Probabilities 

Let  X    be  distributed  binomially  with  parameters  p.  and  n^, 
i  =  l,2,...,k..   Fisher's  statistic  can  be  inverted  to  from  the  100(l-ct)% 
upper  (i.e.,  p.  >  .5,  all  i)  confidence  region  given  by 

S-     =    {(d^,d^,...,d^):      X   -2  £n  L^^^d.)  <X2i,(l^)>- 

1=1 

In  light  of  the  results  of  Section  3.7,  the  significance  levels  should 
be  determined  via  Lancaster's  mean  chi-square  approximation.  That  is, 
define 

-2  iln[L^^\d.)]  =  2  -  2{P.  ^n  P.  -  P.^^  ^n  ^ ^^^} / C^ ^  -  ^ ^+i'> 

where 

n 
i      ri-    i       "-!~J 

P.  =  z  ,.,  (  .'■)  d!(l-d.) 

The  concept  of  inverting  a  test  combination  procedure  to  obtain 
confidence  regions  is  not  new  (though  perhaps  unrecognized)  because  the 
usual  rectangular  confidence  regions  are  in  fact  inversions  of  the  test 
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combination  statistic  T    .   Since  Fisher's  combination  procedure,  T 
generally  performs  better  than  T    over  a  broad  alternative  space, 

there  is  a  corresponding  benefit  in  constructing  S   instead  of  S^  as 

(F) 
an  omnibus  procedure.   Regions  obtained  from  inverting  T    are  typ- 
ically similar  to  regions  obtained  by  inverting  the  likelihood  ratio 

statistic.   The  rectangular  form  of  S   is  more  easily  reported  than 

m 

is  the  form  of  S  .   However,  S  can  be  displayed  graphically  for  a 

k-dimensional  parameter  with  about  the  same  amount  of  difficulty  that 

is  required  to  display  the  contours  of  a  second-order  response  surface 
in  k  variables. 


4.3   The  Combination  of  2  x  2  Tables 

As  noted  in  Section  3.2,  the  problem  of  combining  independent 
binomial  experiments  is  closely  related  to  the  problem  of  combining 
independent  2x2  tables.   In  the  latter  problem,  each  individual  exper- 
iment consists  of  two  "sub-experiments";  one  to  assess  the  performance 


o 


f  a  standard  treatment,  the  other  to  assess  the  performance  of  an 
experimental  treatment.   The  quantities  of  interest  for  the  i   exper- 
iment are  summarized  in  the  following  listing. 

Experimental      Standard  Total 

.(i)            Y^i)  s^^^ 

Successes       X               i  -^ 

(i)    (i)       (i)    (i)  ,(i)   c(i) 

Failures      n-x         n-y  n      ^^ 

e  s 

(i)             (i)  J^^ 

Total           n               n  n 

e  s 


Ill 


In  typical  cases,  the  experimental  and  standard  treatments 

are  evaluated  with  distinct  groups  of  experimental  units.   It  follows 

that  X    and  Y    are  independent  binomial  random  variables  based  on 

n    and  n    trials  and  with  unknown  success  probabilities  p    and 
e        s  e 

p  ,  respectively. 

It  is  often  desired  to  form  a  combined  test  of  the  null 

hypothesis  H:  p    =  p    ,  i  =  l,2,...,k;  that  is,  that  the  experimental 

and  standard  treatments  are  equivalent  in  all  k  experiments.   Analogous 

to  the  binomial  combination  problem  considered  in  Chapter  III,  there  is 

no  uniformly  most  powerful  test  versus  the  general  alternative 

H  :  p    S  p   "^  (strict  inequality  for  at  least  one  i)  .   The  parametric 
A   e     s 

combination  procedures  discussed  in  Chapter  III  can  be  applied  to  the 
problem  of  combining  2x2  tables  as  shown  in  the  following  paragraphs. 
Define  the  likelihood  function  of  the  i    experiment  to  be 

L(Pe   'Ps   )-l^X^'V^Pe   ^     (1-Pe   -> 

(g(ps )     ^-Ps  ^ 


^':\  f-r 


,(i)'V,(i) 


e       e 


(i) 


-  s       s  -> 


(i) 


(i)        n^^^      ^(i) 

.  (i),-   ^(i)^    ,,         (i)  ^^-n^'^  ^ 
(p   /1-P-  )    (1-Pe  )  (1  Ps   ^ 


If  a  uniformly  most  powerful  test  of  H.:  p    = p^   versus 
-'       "^  1    e     s 


A:  p    >P    exists  for  the  i  ^  individual  experiment,  it  is  of  the  form 
e     s 


,,  (i)  _   (i)  _  „(i). 
L(Pg   =  P^,   -  P   ) 

L(p^    >  p^   ) 


<  c 
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which    is    equivalent    to 

(p   /1-P 2 ikiB. 1 


—  <  c, 


(i)      ..^  n_^\.        (i)  "s 


s        s 


a)/i_p(i))S^  \i-p(i))  -  (i-P^^) 


(p-./,_p. .)    (l_p^ 


,(i) 


.(i) 


Conditional  on  S^   ,  this  is  equivalent  to  X    >c.   This  is  known  as 
Fisher's  exact  test.   Under  H.,  X^^^  has  a  hypergeometric  distribution 
(conditional  on  S^^^).   If  Fisher's  statistic  is  to  be  used  to  form  a 
combined  test,  the  individual  significance  levels  L    ,  i  =  l,2,...,k, 
should  be  determined  via  Fisher's  exact  test.   As  noted  in  Chapters  I 
and  II,  if  the  individual  significance  levels  are  determined  by  optimal 
tests,  then  Fisher's  procedure  yields  an  optimum  combination  procedure 
in  the  Bahadur  sense. 

The  likelihood  function  for  all  k  experiments  is 

k 


..  (1)   (1)   ,  (2)   (2)       ,  (k)  ik)  U;   UK 

L((p    ,p   .,  (p^   ,P^  ),...,{?        ,P        ))  -    11  (.P    .P    ) 


(i)   (i) 


i=l 


If  a  uniformly  most  powerful  test  of  the  combined  null  hypothesis 

H:  p^^^  =  p^^\  all  i,  versus  the  alternative  H  :  pj"   >  p/   (strict 
e      s  A   e      b 

inequality  for  at  least  one  i)  exists  it  is  of  the  form 


(i) 


<  c, 


s       s  e  s 
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Conditional  on  (S^  ■* ,  S    ,...,S^  ')    this  is  equivalent  to 

I   X^^)  >  c 
i=l 

if  the  odds  ratio, 

(i),,     (i) 
P   /I  -  P 

^i    (i)/,     (i) 
P   /I  -  P 

5  S 

k 

is  constant  on  i.   The  test   E  X  ^   >  c  is  equivalent  to  the  Cochran- 

1=1 

Mantel-Haenzsel  procedure  given  in  (3.4).   In  other  words,  for  the 

(  PMH  ^ 
combination  of  2  x 2  tables,  T      is  uniformly  most  powerful  for 

H:  ^1  =  ^2  "  '■■  "  ^k  "  ""^  versus  H^:  ^j^  =  ^2  "  '"  "  ^k  ^  ■'" 

The  test  statistics  t''^^\    t^^^^\    T^™\  and  T^^^  can  also  be 
constructed  to  yield  combined  tests  for  the  2x2  tables  problem. 
A  comparison  of  the  procedures  parallel  to  that  given  in  Chapter  III 
is  indicated.   In  particular,  the  relative  performance  of  the  proce- 
dures for  various  points  in  the  parameter  space  {(, ^,1.^,  .  .  .  ,^^)    needs  to 
be  determined.   As  in  Chapter  III,  the  notion  of  Bahadur  asymptotic 
relative  efficiency  could  be  used  and  exact  power  studies  could  be 

undertaken. 

4.4   Testing  for  the  Heterogeneity  of  Variances 

SuppoKc  X,  ,x,.  .  .  .  ,x   ,iro  muLunlly  i  ndci'riuliMU  normal  random 
'  '       12       in 

2   2       2       . 
variables  with  means  M  ,M ,,,...  ,y^^^  and  variances  "j^'°2'""°m'      ^^ 

2 
often  desired  to  test  for  the  heterogeneity  of  the  o^.   The  likelihood 
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tio  test  was  proposed  as  a  solution  by  Neyman  and  Pearson  [32]. 
Bartlett  [5]  subsequently  proposed  a  modification  to  the  likelihood 
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ratio  test.   A  further  simplification  leads  to  the  maximum  F  test  due 
to  Hartley  [15].   The  relative  merits  of  these  procedures  are  discussed 
by  Hartley.   None  of  these  three  well-known  methods  yield  statistics 
with  common  distributions  and  therefore  require  either  special  tables 
or  approximation  procedures. 

An  alternative  test  can  be  constructed  by  applying  Fisher's  combi- 
nation method  to  a  sequence  of  independent  statistics  described   by 
Hogg  [16].   Hogg  points  out  that  the  ratios 

V. 


,   i  =  2,3,... ,m 


R.  =  ^ 

X      1 

S   V. 
J=l  ' 

where  the  V   are  the  sample  variances  are  stochastically  independent 
i 

2    2  2 

under  the  null  hypothesis  H  :  o.  =  o  =  . . .  =  o  .   Thus,  when  multi- 
•  ^  o    1     2  m 

plied  by  the  appropriate  constants,  the  R.  yields  a  succession  of 

m-1  mutually  independent  F-tests.   The  resultant  significance  level 

(F) 
can  than  be  combined  by  the  statistic  T    to  yield  a  test  of  H^. 

The  computations  involved  in  this  alternative  approach  are 
probably  more  cumbersome  than  even  Bartlett's  test.   From  theoret- 
ical considerations  presented  in  this  paper,  however,  it  is  likely  that 
the  performance  of  this  alternative  method  will  approximate  that  of 

the  likelihood  ratio  procedure.   An  added  benefit  is  that  the  null 

(F) 
distribution  of  T    is  known  exactly. 
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4.5   Testing  for  the  Difference  of  Means  with  Incomplete  Data 

Suppose  (X^ ,X) '  is  a  random  vector  normally  distributed  with 
mean  vector  (M,,P„)'  and  covariance  matrix  |.   Assume  that  N  pairs  of 
observations  are  taken  on  (X  ,X  )  '  of  which  N-n  observations  corre- 
sponding to  X^  are  randomly  missing  and  N  -  n„  observations  correspond- 
ing to  X„  are  randomly  missing.   Testing  H  :  P   =  U™  versus  H  :  U^  > U^ 
can  be  described  as  the  usual  paired  t-test  with  missing  data. 
Special  cases  of  this  problem  are  considered  in  detail  by  Lin  [23,24,25]. 
The  interested  reader  is  referred  to  Lin  for  additional  references. 
To  date,  the  work  of  Lin  is  based  on  the  assumption  that  data  are  miss- 
ing only  for  one  variate.   (He  notes  in  [25]  that  results  for  the  more 
general  case  of  data  missing  on  both  variates  are  forthcoming.) 

For  the  case  when  \    is  not  known  and  no  data  are  discarded, 
only  approximate  tests  have  been  derived,   Lin  sliows,  however,  that  in 
most  cases  the  approximate  tests  are  more  powerful  than  the  exact  tests 
formed  by  discarding  data. 

An  alternative  approach  to  the  problem  described  in  the  first 
paragraph  of  this  section  is  to  apply  Fisher's  combination  procedure. 
A  description  of  this  approach  follows.   Consider  two  subsets  of  the 
original  data  set;  (1)  the  N-n  -n   complete  pairs,  and  (2)  the  n,  un- 
matched observations  on  X   and  n   unmatched  observations  on  X  .   The 
data  in  subset  1  can  now  be  analyzed  via  the  usual  paired  t-test.   The 
data  in  subset  2  can  be  analyzed  via  the  usual  two-sample  t-test.   The 
statistics  that  result  are  independent  since  they  are  based  on  distinct 
sets  of  data.   Combination  of  the  concomitant  significance  levels  yields 
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an  exact  method  of  performing  the  hypothesis  test  of  interest.   The 
optimality  results  of  T^^^  established  thus  far  in  this  paper  tend  to 
suggest  that  this  approach  will  yield  a  viable  solution. 

A  possible  added  advantage  to  the  approach  described  here  is 
its  simplicity.   The  same  idea  can  be  generalized  to  more  complex 
designs  where  missing  data  have  occurred. 

4.6   Asymptotic  Efficiencies  for  k->°° 

The  Bahadur  efficiencies  given  in  this  dissertation  are 
derived  assuming  a  fixed  number  of  tests,  k,  are  performed.   The  sample 
sizes  are  allowed  to  increase  without  bound.   Experimental  situations 
often  occur  in  which  an  individual  experiment  is  replicated  several 
times  with  the  sample  size  remaining  fairly  constant.   For  such  cases, 
the  derivation  of  Bahadur  efficiencies  may  be  more  appropriate  if  it  is 
assumed  that  each  experiment  is  based  upon  a  fixed  sample  size,  n,  and 
allowing  k  to  increase  without  bound. 

Monti  and  Sen  [30]  consider  the  case  k-»-°°  in  their  approach  to 
the  combination  problem.   They  derive  locally  optimum  tests  for  a  gen- 
eral multivariate  framework.   For  local  alternatives,  Monti  and  Sen 
show  that  their  procedures  are  more  efficient  in  the  Bahadur  sense. 

The  Bahadur-Savage  Theorem  (Theorem  1)  can  be  used  to  derive 
the  Bahadur  slopes  of  the  various  combination  procedures  for  the  case 
k-><=°.   In  order  to  derive  the  function  f(t)  for  Part  2  of  the  Bahadur- 
Savage  Theorem,  Chernoff's  Theorem  (Theorem  3)  may  be  useful.   If  the 
individual  tests  are  based  on  a  common  sample  size,  the  statistics 
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formed  by  the  combination  procedures  will  he  sums  of  independent, 
identically  distributed  random  variables.   If  the  moment  generating 
function  of  an  individual  test  statistic  is  known,  Chernoff's  Theorem 
can  then  be  directly  applied. 
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